From 6e4dd77a6ab223739c49cad2bf793048e1b5ab79 Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 10 Nov 2026 05:00:27 +0000 Subject: [PATCH 1/2] Simplify the docs to the current approach and the key decisions The docs had grown to about 40k words across 31 files, with the same rules restated in the invariants, the ADRs, both specs, the architecture map and the plain-language tour. They now run to about 12k words in 10 files, and each fact has one home. - invariants.md keeps every id (S1 to K1) and states each rule in a few lines. - decisions.md replaces ADRs/: nine decisions that are expensive to reverse, with the reason for each. ADR numbers stay as stable ids. The other thirteen records restated an invariant and are retired. - architecture.md absorbs index-engine.md and the mechanics from search-and-similarity.md: crates, the index, and the flows. - data-model.md is the on-disk contract only: notes, links, verbs, resources. - evals.md keeps the instruments, the gate, the rulings and the process rules (same numbers). - quickstart.md keeps the walkthrough, every command (adds `b2 why`) and every environment variable, and now explains the strength dots. - CLAUDE.md and the desktop CLAUDE.md are tightened, and gain the rules that keep the docs short: one home per fact, no history in docs, no section numbers in citations. Removed: ADRs/, docs/README.md, docs/index-engine.md, docs/search-and-similarity.md. Co-Authored-By: Claude Claude-Session: https://claude.ai/code/session_01PgG847TLizZXGt5hVNqiKm --- ADRs/0001-design-docs-are-normative.md | 25 - ...-truth-index-is-a-disposable-projection.md | 30 - ...ote-identity-is-the-vault-relative-path.md | 26 - ...-markdown-is-the-only-surface-b2-writes.md | 27 - ADRs/0005-ai-behind-enumerated-seams.md | 28 - ...ctors-in-plain-tables-content-addressed.md | 29 - ...ding-space-identity-includes-the-device.md | 24 - ...retrieval-bm25-plus-vector-fused-by-rrf.md | 24 - ...discovery-is-model-free-at-surface-time.md | 25 - ...typed-graph-two-homes-closed-vocabulary.md | 34 - ADRs/0011-no-async-runtime-in-b2-code.md | 29 - ...0012-one-typed-facade-two-dumb-adapters.md | 29 - ...013-model-quality-is-measured-out-of-ci.md | 34 - ...scovery-always-serves-the-ranked-prefix.md | 63 -- ...ed-search-result-is-a-claim-of-evidence.md | 54 -- ...dering-note-content-is-a-trust-boundary.md | 26 - ...yboard-complete-with-one-chord-registry.md | 35 - ADRs/0018-ci-runs-make-ci-and-nothing-else.md | 37 - .../0019-build-our-own-sqlite-index-engine.md | 43 -- ...020-embeddings-inside-the-single-binary.md | 31 - ...ency-is-serialized-on-sqlites-own-locks.md | 49 -- ...am-carries-tool-calls-for-bounded-turns.md | 57 -- ADRs/README.md | 36 - CLAUDE.md | 225 +++--- README.md | 82 +-- crates/b2-desktop/CLAUDE.md | 189 +++-- crates/b2-embed/evals/README.md | 6 +- docs/README.md | 33 - docs/architecture.md | 385 +++++----- docs/data-model.md | 574 +++------------ docs/decisions.md | 133 ++++ docs/evals.md | 599 ++++------------ docs/index-engine.md | 658 ------------------ docs/invariants.md | 484 ++++--------- docs/quickstart.md | 549 ++++----------- docs/search-and-similarity.md | 363 ---------- fixtures/README.md | 56 +- 37 files changed, 1124 insertions(+), 4007 deletions(-) delete mode 100644 ADRs/0001-design-docs-are-normative.md delete mode 100644 ADRs/0002-vault-is-truth-index-is-a-disposable-projection.md delete mode 100644 ADRs/0003-note-identity-is-the-vault-relative-path.md delete mode 100644 ADRs/0004-markdown-is-the-only-surface-b2-writes.md delete mode 100644 ADRs/0005-ai-behind-enumerated-seams.md delete mode 100644 ADRs/0006-vectors-in-plain-tables-content-addressed.md delete mode 100644 ADRs/0007-embedding-space-identity-includes-the-device.md delete mode 100644 ADRs/0008-hybrid-retrieval-bm25-plus-vector-fused-by-rrf.md delete mode 100644 ADRs/0009-discovery-is-model-free-at-surface-time.md delete mode 100644 ADRs/0010-typed-graph-two-homes-closed-vocabulary.md delete mode 100644 ADRs/0011-no-async-runtime-in-b2-code.md delete mode 100644 ADRs/0012-one-typed-facade-two-dumb-adapters.md delete mode 100644 ADRs/0013-model-quality-is-measured-out-of-ci.md delete mode 100644 ADRs/0014-discovery-always-serves-the-ranked-prefix.md delete mode 100644 ADRs/0015-a-served-search-result-is-a-claim-of-evidence.md delete mode 100644 ADRs/0016-rendering-note-content-is-a-trust-boundary.md delete mode 100644 ADRs/0017-the-gui-is-keyboard-complete-with-one-chord-registry.md delete mode 100644 ADRs/0018-ci-runs-make-ci-and-nothing-else.md delete mode 100644 ADRs/0019-build-our-own-sqlite-index-engine.md delete mode 100644 ADRs/0020-embeddings-inside-the-single-binary.md delete mode 100644 ADRs/0021-concurrency-is-serialized-on-sqlites-own-locks.md delete mode 100644 ADRs/0022-the-chat-seam-carries-tool-calls-for-bounded-turns.md delete mode 100644 ADRs/README.md delete mode 100644 docs/README.md create mode 100644 docs/decisions.md delete mode 100644 docs/index-engine.md delete mode 100644 docs/search-and-similarity.md diff --git a/ADRs/0001-design-docs-are-normative.md b/ADRs/0001-design-docs-are-normative.md deleted file mode 100644 index a23165c..0000000 --- a/ADRs/0001-design-docs-are-normative.md +++ /dev/null @@ -1,25 +0,0 @@ -# ADR-0001 — Design docs are normative; the code is a projection of them - -- **Status:** Accepted · 2026-06-28 -- **Refs:** `docs/invariants.md` - -## Context - -B2's behaviour is easy to drift by accident: a schema tweak or a threshold change can quietly -redefine what the product promises. Prose that merely *describes* the code cannot stop that. - -## Decision - -- `docs/invariants.md` is the **normative register** — one testable claim per entry, cited - by id (S2, G2, D1…). **On conflict with any other doc or with the code, the register wins** and - the other side gets fixed. -- `data-model.md` is the *what*, `index-engine.md` the *how*. Code comments cite them by section. -- Changing the register is a deliberate decision (an ADR here + the issue that drove it), never a - drive-by edit. - -## Consequences - -- Before changing behaviour, read the relevant doc: the schema must satisfy the data model, never - the reverse. -- Decision history is the GitHub Issue that drove a verdict plus the commit that shipped it. These - ADRs record only the *architectural* choices; the backlog stays in Issues, build history in git. diff --git a/ADRs/0002-vault-is-truth-index-is-a-disposable-projection.md b/ADRs/0002-vault-is-truth-index-is-a-disposable-projection.md deleted file mode 100644 index 6b59557..0000000 --- a/ADRs/0002-vault-is-truth-index-is-a-disposable-projection.md +++ /dev/null @@ -1,30 +0,0 @@ -# ADR-0002 — The vault is the source of truth; the index is a disposable projection - -- **Status:** Accepted · 2026-06-28 -- **Refs:** invariants S1–S5 · `index-engine.md` §3, §8 - -## Context - -An AI layer over personal notes can either own the notes (a database with an export button) or sit -beside them. Owning them is lock-in, and it makes every index bug a data-loss risk. - -## Decision - -- **`index = a pure projection of (the vault directory)`.** Two tiers: the vault (Markdown + - resources + the directory tree) is authoritative; `/.b2/b2.sqlite` is a cache. -- Delete the index, `reindex`, get an identical index. **Incremental re-index ≡ full rebuild**, - unconditionally — whole-vault passes own every reconciliation (pruning rows for vanished files, - collecting unreferenced vectors); single-note paths never prune. -- **No durable B2-derived state outside the Markdown** — no event log, no sidecars. -- A schema change is a version bump + rebuild, never a data migration. A migration script would be - evidence this ADR broke. -- Folders are user-authored structure and are **never projected**: the tree listing is a live fs - walk, and `create_dir`/`move_dir`/`delete_dir` proxy the OS. - -## Consequences - -- Any feature needing durable state must find a home in the Markdown or do without. -- Many processes may hold one index open. Readers are unrestricted and never refused; the two - drop-and-rebuild paths (schema migration, vector tables) serialize on SQLite's own write lock, - each checking and re-checking inside `BEGIN IMMEDIATE` (GH #114). The CLI's `reindex` advisory - lock is writer-only and cannot cover this. diff --git a/ADRs/0003-note-identity-is-the-vault-relative-path.md b/ADRs/0003-note-identity-is-the-vault-relative-path.md deleted file mode 100644 index a5944a0..0000000 --- a/ADRs/0003-note-identity-is-the-vault-relative-path.md +++ /dev/null @@ -1,26 +0,0 @@ -# ADR-0003 — A note's identity is its vault-relative path - -- **Status:** Accepted · 2026-08-13 — supersedes the `b2id` ULID stamp -- **Refs:** invariants L1, L3, S3, W1 · GH #170 - -## Context - -Through 2026-08 B2 stamped a `b2id:` ULID into the frontmatter of every note it saw. It bought -re-binding after an out-of-band move, and cost a collision subsystem, a carve-out on the -"incremental ≡ full" invariant, an unbidden write into the user's files — and no link ever named it. - -## Decision - -- Identity is the **vault-relative path**. `notes.path` is the primary key; every derived row - cascades off it. Both link homes already address by path, so the index holds no key the vault does - not carry. -- Notes and resources share this identity model; the remaining asymmetry is authoring surface alone. -- A move **B2 performs** rewrites inbound link path text and re-keys the moved note's rows in one - transaction. A move made **outside** B2 is a delete plus a create — inbound links surface as - dangling, never silently dropped. - -## Consequences - -- No migration exists or is needed: a `b2id:` line left by an older B2 is an ordinary unknown - frontmatter key — never read, never removed. `rm -rf .b2/` and reindex is the whole upgrade. -- Content-addressed vectors (ADR-0006) are what make path identity cheap: a rename re-embeds nothing. diff --git a/ADRs/0004-markdown-is-the-only-surface-b2-writes.md b/ADRs/0004-markdown-is-the-only-surface-b2-writes.md deleted file mode 100644 index 6702dd6..0000000 --- a/ADRs/0004-markdown-is-the-only-surface-b2-writes.md +++ /dev/null @@ -1,27 +0,0 @@ -# ADR-0004 — Markdown is the only surface B2 writes, and B2 makes no unbidden writes - -- **Status:** Accepted · 2026-06-28 -- **Refs:** invariants W1–W5, S2 · `data-model.md` §0–§1 - -## Context - -"Your files stay yours" is only true if it is mechanical. A tool that tidies frontmatter or injects -its own syntax on read has already taken ownership of the vault. - -## Decision - -- **Markdown is the vault's sole authored subset** — the only format whose bytes B2 may write. - Non-`.md` files are *resources*: path-keyed peers contributing derived rows only. -- **Reading writes nothing.** Walking, projecting and reindexing a vault leave a git-versioned vault - with no diff, and run on a read-only one. -- The on-command writes are **enumerated**: `b2 link` appending one frontmatter `b2_relations:` - entry; the move-repair of inbound link *path text*; the editor save (a byte-honest splice of the - human's own body bytes, content-hash guarded); the frontmatter save (same guard, body untouched); - import (handed bytes copied verbatim, then projected); and create/move/delete on explicit command. -- **B2 never authors body content and never asks the body to carry B2 syntax.** Round-trip is - lossless: unknown frontmatter keys survive verbatim and in order; B2's one key is namespaced. - -## Consequences - -- Every committed connection must be expressible as ordinary Markdown a human could have typed. -- Consequences of human edits (orphans, dangling links) are surfaced, never silently repaired. diff --git a/ADRs/0005-ai-behind-enumerated-seams.md b/ADRs/0005-ai-behind-enumerated-seams.md deleted file mode 100644 index 193e490..0000000 --- a/ADRs/0005-ai-behind-enumerated-seams.md +++ /dev/null @@ -1,28 +0,0 @@ -# ADR-0005 — AI lives behind two enumerated seams; the core is model-free - -- **Status:** Accepted · 2026-07 (chat seam 2026-08-14) -- **Refs:** invariants M1, M3, M5, E2 · GH #151, #153, #154 - -## Context - -The Bitter-Lesson tenet: build for tomorrow's model. Machinery that compensates for today's model -becomes the thing blocking a better one — and any model dependency in the engine makes the test -suite slow, non-deterministic, and network-bound. - -## Decision - -- Exactly two seams: **`Embedder`** (text → vector) and **`LlmProvider`** (grounded chat — streamed, - cooperatively cancellable). A reranker, if one lands, is the next enumerated seam, not an exception. -- `b2-core` is **model-free** (never links candle) and is built and tested against deterministic - fakes (`FakeEmbedder`, `FakeLlm`); the real model drops in with no schema or flow change. -- Model-compensating machinery (per-pair adjudication, query expansion, heavy orchestration) is - deferred or default-off. -- One embedding space in v1: every vault member funnels to text through the same model. -- **Note content never leaves the machine unbidden.** The default chat endpoint is local; a cloud one - exists only by explicit configuration, so the consent moment is the configuration moment. - -## Consequences - -- Chat carries **no index identity** (contrast ADR-0007): nothing it produces is stored, history is - session-only, so swapping chat models never touches the index. -- Model quality cannot be tested in CI, which is what the eval harness exists for (ADR-0013). diff --git a/ADRs/0006-vectors-in-plain-tables-content-addressed.md b/ADRs/0006-vectors-in-plain-tables-content-addressed.md deleted file mode 100644 index 1f193ec..0000000 --- a/ADRs/0006-vectors-in-plain-tables-content-addressed.md +++ /dev/null @@ -1,29 +0,0 @@ -# ADR-0006 — Vectors live in plain SQLite tables, scored in-process, content-addressed - -- **Status:** Accepted · 2026-07-12 (content-addressing 2026-08-13) -- **Refs:** invariants M4 · `index-engine.md` §3–§4 · GH #38, #170 - -## Context - -A vector extension (sqlite-vec, FAISS) is another binary to ship, version, and fail to load — for a -personal vault whose vectors number in the thousands, not the millions. - -## Decision - -- `embeddings(text_hash, vector)` and `note_centroids(note_path, centroid)` are **plain BLOB - tables**. Every distance is computed **in-process** in one scan (`embed::l2_sq`). -- The tables are created at **embed time**, not in the base migration: their existence *is* the - "this vault has an embedding space" signal that the fallbacks key on (BM25-only search on a - projected-but-unembedded vault). -- The store is **content-addressed**: the key is blake3 of the chunk text, which is exactly the embed - input. Identical text has one vector. -- Centroids are derived data on the vectors' lifecycle: refreshed by the embed pass, dropped on - re-chunk. No separate invalidation exists. - -## Consequences - -- A moved or renamed note re-embeds nothing — which is what makes path identity (ADR-0003) cheap. -- A vector **outlives** the chunk that addressed it (it may be shared), so `replace_chunks` cannot - cascade it away; the whole-vault pass collects hashes no chunk references. Every read joins through - `chunks`, so an uncollected orphan is invisible to search, discovery, and coverage alike. -- `--force` re-*chunks* but re-embeds only what genuinely differs. diff --git a/ADRs/0007-embedding-space-identity-includes-the-device.md b/ADRs/0007-embedding-space-identity-includes-the-device.md deleted file mode 100644 index c6842e5..0000000 --- a/ADRs/0007-embedding-space-identity-includes-the-device.md +++ /dev/null @@ -1,24 +0,0 @@ -# ADR-0007 — The embedding space has one recorded identity, and the compute device is part of it - -- **Status:** Accepted · 2026-07-13 -- **Refs:** invariants M2 · `index-engine.md` §6 · GH #40 - -## Context - -Mixing vectors from two models — or from two devices whose arithmetic differs — returns -silently-wrong results. Silence is the failure mode that matters: nothing looks broken. - -## Decision - -- `meta.(embed_model_id, embed_dim)` is the **only** place a model swap is detectable. -- The resolved compute device folds into that id: CPU keeps the bare repo id, a `--features metal` - GPU build appends `@metal`. A device change that alters vectors **is** a model swap. -- On a mismatch: `search` **fails fast**; `reindex` drops both vector tables and re-embeds; `open` - **never** mutates the vector space, so changing configuration cannot wipe vectors on the next - command. - -## Consequences - -- Metal is a **build** switch, not a runtime one — recompile to flip, and re-embed the vault. -- Any per-model calibrated constant (ADR-0015's evidence bar) must be keyed to this id, device - suffix included. diff --git a/ADRs/0008-hybrid-retrieval-bm25-plus-vector-fused-by-rrf.md b/ADRs/0008-hybrid-retrieval-bm25-plus-vector-fused-by-rrf.md deleted file mode 100644 index 068dbbe..0000000 --- a/ADRs/0008-hybrid-retrieval-bm25-plus-vector-fused-by-rrf.md +++ /dev/null @@ -1,24 +0,0 @@ -# ADR-0008 — Retrieval is BM25 ⊕ vector KNN, fused by Reciprocal Rank Fusion - -- **Status:** Accepted · 2026-07 -- **Refs:** `index-engine.md` §4 · flow ② in `search.rs` - -## Context - -Lexical search misses paraphrase; vector search misses exact terms, names, and identifiers. A -personal vault needs both, and score-level blending would require calibrating two incomparable scales. - -## Decision - -- Search fuses BM25 over `chunks_fts` with an exact in-process vector KNN over `embeddings`, using - **RRF (k=60)** over ranks, resolved from chunks up to notes. -- Raw natural-language queries are sanitized into a safe FTS5 `MATCH` expression (punctuation is - FTS5 syntax and would otherwise crash the parse). -- On a projected-but-unembedded vault, search degrades to BM25-only rather than failing. - -## Consequences - -- RRF discards absolute signals, which is why a query-level evidence reading had to be carried - alongside the fused order rather than derived from it (ADR-0015). -- Candidate width is a real tuning axis invisible to a small labelled corpus; the model-free rank - probe exists for it (ADR-0013). diff --git a/ADRs/0009-discovery-is-model-free-at-surface-time.md b/ADRs/0009-discovery-is-model-free-at-surface-time.md deleted file mode 100644 index d384b63..0000000 --- a/ADRs/0009-discovery-is-model-free-at-surface-time.md +++ /dev/null @@ -1,25 +0,0 @@ -# ADR-0009 — Discovery surfaces candidates; the human commits the link - -- **Status:** Accepted · 2026-07-12 -- **Refs:** invariants G1, D1 · `index-engine.md` §3, §5 · GH #38 - -## Context - -The obvious design is an inbox of machine-proposed links with accept/reject state. That makes the -index authoritative for something (ADR-0002), adds a lifecycle to edges, and asks the user to -maintain a queue. - -## Decision - -- **`b2 similar`** ranks the semantically nearest *unlinked* notes in **two stages**: a coarse - O(notes) scan over `note_centroids` shortlists candidates, then exact max-sim over only that - shortlist's chunk vectors, minus the anchor's 1-hop graph neighbours. **No model call at surface - time.** -- **`b2 link`** (or, in the GUI, dragging a card onto a line to type a `[[wikilink]]`) is the human - committing. A connection exists only once authored. -- **There is no suggestion queue**, no pending state, and nothing inert in the graph. - -## Consequences - -- The human is the precision gate, which is what lets discovery favour recall. -- Ranking quality is a surfacing question, not a storage one — see ADR-0014. diff --git a/ADRs/0010-typed-graph-two-homes-closed-vocabulary.md b/ADRs/0010-typed-graph-two-homes-closed-vocabulary.md deleted file mode 100644 index 0c2c45e..0000000 --- a/ADRs/0010-typed-graph-two-homes-closed-vocabulary.md +++ /dev/null @@ -1,34 +0,0 @@ -# ADR-0010 — The typed graph: two authored homes, frontmatter-wins, a closed verb core - -- **Status:** Accepted · 2026-07 -- **Refs:** invariants G1–G6 · `data-model.md` §0–§3 - -## Context - -Typed relations need somewhere to live that survives B2 being deleted, without turning note prose -into a markup dialect. The verb set has two consumers — the human typing a connection, and queries -(`b2 neighbors --type supports`) — and both want the core small, orthogonal, and stable, so the same -relationship always gets the same verb. The core encodes the one thing embedding similarity cannot -infer: **stance**. The model already surfaces "these are related"; whether the notes *agree* is what -only the human at the typing moment knows. - -## Decision - -- The edge set is the union of exactly two homes: **body links** (`origin=inline`, always untyped - `references` — the body carries no B2 syntax, so no verb or explanation is ever parsed from prose) - and **frontmatter `b2_relations:`** (`origin=frontmatter`, the sole home of a verb + explanation). -- Same `(target, type)` in both homes → **frontmatter wins** (it alone can carry the explanation); a - *different* verb over a body-linked target coexists (the augment case). -- The vocabulary is a **closed three-verb stance core** — `references` / `supports` / `contradicts` — - plus a tolerated tail stored verbatim. -- Edges are **directed and stored once**, with a deterministic id from `(src, dst, type, occurrence)`. - Inverse labels are display-only; B2 never writes a reciprocal link into the target file. -- There is **no `status` column**: every edge is authored and active. An unresolvable target projects - as a surfaced dangling edge, never a dropped one, and heals on the next reindex. - -## Consequences - -- The `edges` table is a **cache**; runtime parsing is the correctness definition. It exists for what - parsing cannot serve — backlinks, typed traversal, the discovery exclusion — and is rebuilt from - scratch every reindex. -- Nothing is ever copied between homes or auto-removed from a file. diff --git a/ADRs/0011-no-async-runtime-in-b2-code.md b/ADRs/0011-no-async-runtime-in-b2-code.md deleted file mode 100644 index be5d448..0000000 --- a/ADRs/0011-no-async-runtime-in-b2-code.md +++ /dev/null @@ -1,29 +0,0 @@ -# ADR-0011 — No B2 code is async; no B2 crate starts a runtime - -- **Status:** Accepted · 2026-08-14 -- **Refs:** GH #154, #174 · `make no-tokio` - -## Context - -B2 is a local single-binary tool whose I/O is a file walk, a SQLite handle, and one HTTP stream at a -time. An async runtime buys nothing here and costs a large dependency tree — and the claim "no tokio -in the workspace" was once made of the lockfile and was **false** there: `hf-hub`'s default features -pulled reqwest → hyper → tokio into the `b2` binary to serve a path nothing called. - -## Decision - -- Every B2 crate is **sync end-to-end**. Cancellation is returning early from a blocking read loop, - not a future being dropped. -- `b2-llm` is a hand-rolled sync OpenAI-compatible SSE client over `ureq`; `hf-hub` is taken - `default-features = false` so B2 uses its sync API (TLS is rustls + webpki-roots, the same stack - `b2-llm` uses). -- **`make no-tokio` is a gate** — it reads the lockfile, compiles nothing, and leads `make ci` - because a dependency added with default features on can restore the whole async HTTP stack while - every other stage stays green. -- Do not introduce `async` (or generics, traits, macros) without a concrete need. - -## Consequences - -- The one runtime in the tree is **Tauri's**, in the desktop host, which is what a GUI host is. -- Streaming cancellation is a callback return value (`Break`) read at token granularity, in both - adapters (Ctrl-C in the CLI, Esc in the GUI). diff --git a/ADRs/0012-one-typed-facade-two-dumb-adapters.md b/ADRs/0012-one-typed-facade-two-dumb-adapters.md deleted file mode 100644 index 2705df7..0000000 --- a/ADRs/0012-one-typed-facade-two-dumb-adapters.md +++ /dev/null @@ -1,29 +0,0 @@ -# ADR-0012 — One typed façade, two dumb adapters - -- **Status:** Accepted · 2026-07 -- **Refs:** invariants E3, E4 · `b2-core/src/vault.rs` · `crates/b2-desktop/CLAUDE.md` - -## Context - -Two front ends (a CLI and a Tauri app) over one engine will grow two subtly different products -unless the boundary is enforced. - -## Decision - -- **`Vault` is the one typed API.** The CLI and the desktop host are its only clients; every other - `b2-core` module is called directly only by integration tests. -- Each adapter command is **deserialize → one façade call → serialize**. The desktop host reuses the - CLI's `--json` view types as its IPC contract. Logic that wants to live in an adapter belongs - behind the façade. -- Dependencies point one way (adapters → core, never back). Adapters own only what the host demands: - clocks, log subscribers, the OS dialog, the menu bar, the cancellable background task. -- **Façade operations are added when a command needs them** — never a pre-built broad surface. -- The embedder (and chat provider) are **injected here**: `open` defaults to the fake, - `open_with_embedder` is how an adapter wires the real model. -- User-facing errors are generic and actionable, never leaking sqlite/io/serde internals; full detail - goes to logs or `B2_DEBUG`. - -## Consequences - -- `b2-core` stays deterministic and wall-clock-free (no randomness, timestamps injected). -- The desktop crate's low unit-coverage number is by design. diff --git a/ADRs/0013-model-quality-is-measured-out-of-ci.md b/ADRs/0013-model-quality-is-measured-out-of-ci.md deleted file mode 100644 index eaf14ed..0000000 --- a/ADRs/0013-model-quality-is-measured-out-of-ci.md +++ /dev/null @@ -1,34 +0,0 @@ -# ADR-0013 — Model quality is measured out of CI, by a labelled harness - -- **Status:** Accepted · 2026-07-13 -- **Refs:** invariants E2 · `docs/evals.md` (the process rules) · GH #44, #141, #187 - -## Context - -`cargo test` can prove `reindex` is idempotent; only a human-labelled corpus can say that "how do -leaves turn light into food" should rank `photosynthesis.md` first. Putting that in CI would make -the gate slow, networked, and flaky. - -## Decision - -- **Quality never enters CI.** Real-model work lives behind `b2 init` and the harness under - `crates/b2-embed/evals/`, run on demand (`make eval`, `make eval-sweep`, `make eval-stemmer`, - `make eval-chat`, `make eval-metal`). -- Two halves that rule together: `make eval` scores **quality** (it can say *better*) on labelled - corpora; `make stability` is a model-free rank probe on a ~200-note fixture that scores - **movement** (it can only say *different*) — and sees candidate-width changes a small corpus is - structurally blind to. -- **Constants in code, measurements in the harness.** A shipped threshold's justification is - **re-derived on every run** by the harness rather than quoted in a comment, because a number - frozen into a comment goes stale the first time the corpus grows a shape it never saw. -- **A distributional constant owes a transfer check** (`make calibrate VAULT=`) on a real vault - before it ships: a constant read off one corpus's distribution describes that corpus. -- `#[ignore]` is forbidden — a check that genuinely needs the real model belongs in the harness, - where it actually runs. - -## Consequences - -- Exit-gate assertions are set a margin to the *safe* side of the day's reading, so headroom absorbs - corpus drift rather than run noise (repeat runs are bit-identical). -- Corpus edits ship as their own commit plus a two-direction token audit; a paired per-query - win/loss list is the primary A/B readout. Numbers live in `results.jsonl` (gitignored, local). diff --git a/ADRs/0014-discovery-always-serves-the-ranked-prefix.md b/ADRs/0014-discovery-always-serves-the-ranked-prefix.md deleted file mode 100644 index 159f391..0000000 --- a/ADRs/0014-discovery-always-serves-the-ranked-prefix.md +++ /dev/null @@ -1,63 +0,0 @@ -# ADR-0014 — Discovery always serves the ranked top-N; no anchor-local existence gate - -- **Status:** Accepted · 2026-08-18 (bake-off closed 2026-08-22) — supersedes the z-score - `DiscoveryFloor` -- **Refs:** invariants D1 · GH #182, #192, #196, #197, #200 - -## Context - -Discovery shipped with a **z-score existence gate**: a candidate had to stand out from its anchor's -own pool to be served. Dogfooding a real **single-domain** vault broke it in one reading — with no -unrelated tail, every leader's z compresses under the bar, and **16 of 17 panes went dark on a vault -whose ranking was correct throughout**. The defect was the gate, not its calibration: a -single-population outlier test cannot distinguish *nothing is related* from *everything is related*. - -**The evidence trail**, kept as citations rather than narrative: - -| Issue | What it measured | Verdict | -|---|---|---| -| [#150](https://github.com/AlteredCraft/B2/issues/150) | absolute cosine floors vs. drop-off-from-top-1, against labelled score piles | both fail; shipped a per-anchor **z-score floor** instead | -| [#183](https://github.com/AlteredCraft/B2/issues/183) | a **multi-topic note family** + a **per-mate** metric (the per-anchor one saturates at an anchor's easiest mate) | the centroid-z order both demoted and **suppressed** labelled mates — 3 of 14 never served | -| [#187](https://github.com/AlteredCraft/B2/issues/187) | every candidate's ungated z, with both admissible windows re-derived each run | the **member bar has no window**: mates from +0.80, strangers to +1.62 | -| [#189](https://github.com/AlteredCraft/B2/issues/189) | the journal/daily-note archetype (N≥6 unrelated sections) | averaging disagreeing chunk vectors collapses the centroid toward the corpus mean — the note tops *loner* anchors while its own gem is suppressed | -| [#192](https://github.com/AlteredCraft/B2/issues/192) | judging **after stage 2, on the best-passage z** — the number that orders the list | the unit separates what the centroid could not; 14/15 mates, 0 strangers | -| [#182](https://github.com/AlteredCraft/B2/issues/182) | the desktop band, still calibrated on the *centroid* z after the reorder | bands re-read in the judged unit; the standing rule below | -| [#196](https://github.com/AlteredCraft/B2/issues/196)/[#197](https://github.com/AlteredCraft/B2/issues/197) | the first real vault dogfooded: single-domain, 17 notes | **16 of 17 panes dark on correct rankings** — the gate retired | - -The finding that generalizes: a z-score treats the anchor's population mean as a *noise floor*, valid -only when related notes are rare outliers in a dominant unrelated tail. A single-domain vault has no -such tail — the mean *is* "moderately related" — so the test reads *everything is related* as -*nothing is*. "One diffuse cloud" and "a coherent single-subject vault" are the same geometry from -opposite ends. Large vaults reproduce it inside their own shortlist: above `SHORTLIST_MIN` the judged -population is the anchor's centroid-nearest slice, pre-selected related. - -## Decision - -- **Ranking, reachability, and default disclosure are three separate questions.** Ranking is - relative (best-passage distance); the full ranked list stays **reachable**; `limit` is a cap that - under-fills only for want of scorable notes. -- **No anchor-local statistic may make a candidate unreachable.** The existence gate is retired, not - re-tuned; empty states say only "nothing to compare". -- The z survives **ungated**, as the strength band's input. The band is a within-list grading and - gates nothing. -- A quality signal **may** set the default **disclosure boundary** later, but only as a *prefix* of - the ranked order (row order, band, and fold must never visibly disagree), with everything below one - gesture away — so a misjudged fold costs a keystroke where the retired gate cost the feature. Such - a signal must win a measured bake-off on the orthogonal corpus, the dense single-domain fixture - (where a non-empty default view is absolute), and real vaults — with **"no fold at all" an - admissible winner**. -- **The first bake-off found none** (mutual-kNN reciprocity, GH #200): its admissible window is - empty in both directions, and the two corpora's safe depths are the same *fraction* of their - candidate pools rather than the same constant — a reciprocity depth is a rank in a population and - transfers no better than the cosine and z constants already retired. The block stays in `make eval` - reporting-only, to price the next candidate. - -## Consequences - -- The judged statistic is the same number that orders the list and paints the band, so the three - cannot disagree. **A change to the judged statistic is a change to every surface that paints it** — - the standing rule (GH #182: reordering left the desktop's bands calibrated in the old unit, and its - top band unreachable). -- With no fold, honesty rides on the strength band and the empty-state copy; the dogfood complaint - that opened GH #200 stays **unpaid** on the discovery axis. -- The harness keeps a **structural-zero tripwire** that re-arms on its own if a fold ever ships. diff --git a/ADRs/0015-a-served-search-result-is-a-claim-of-evidence.md b/ADRs/0015-a-served-search-result-is-a-claim-of-evidence.md deleted file mode 100644 index cb3e1cb..0000000 --- a/ADRs/0015-a-served-search-result-is-a-claim-of-evidence.md +++ /dev/null @@ -1,54 +0,0 @@ -# ADR-0015 — A served search result is a claim of evidence - -- **Status:** Accepted · 2026-08-22 -- **Refs:** invariants D2 · GH #201, #202, #206 - -## Context - -Flow ②'s vector half always has *k* nearest — nearest is a fact about the vault, never evidence -about the query — and RRF fuses ranks, discarding the absolute signals that could tell the -difference. As first shipped, the nonsense query `shjfasd` served ten confident-looking results. The -defect was structural, not a loose threshold. - -## Decision - -- `hybrid_search` returns a `Retrieval`: the **same fused order**, plus per hit its rank in each list - and its own distance, plus a query-level reading. `Vault::search_evidence` serves exactly the rows - `search` does, in the same order. -- The rule is **lexical OR semantic** — two independent signals, precisely so it can tell "nothing - matches" from "everything matches", which one signal cannot. -- The lexical half is **IDF-weighted term coverage** (how much of the query's own weight the vault - carries), *not* a df ceiling: the ceiling classed `drone` and `comb` as stopwords in a beekeeping - vault and cut 3 of 15 queries naming notes the vault holds. A fraction of chunks is scale-free in a - vault's size but not in its topical concentration. **The rule changed; the number was not re-tuned.** -- The semantic half is a cosine bar **keyed to `embed_model_id`** (device suffix included), calibrated - in the harness and re-derived every run. -- The verdict is **three-state, each state a different behaviour**: - - **evidence found** → serve as always; - - **no evidence** → the honest "no matches" empty state and **none** of the rows — *strict*: no - reveal, no `--all`, no expander, since a fold is still a surface putting the rows forward, and no - disclosure boundary exists to put them behind (ADR-0014); - - **no calibrated bar for the active model** → serve as always, never "no matches" — the fake - embedder and every unmeasured model land here, and folding this into "no evidence" would blank a - dev vault. - -## Consequences - -- `b2 search --json` is now an **object** (rows + verdict) — a documented break of the array - contract, since a query-level verdict has nowhere to live in a list of rows. It keeps serving the - rows at `vouched: false` where human surfaces show none: an agent handed rows *plus* a verdict can - be honest about them; a reader given rows alone cannot. -- The frontend applies the rule at **one** boundary (`doSearch` drops the rows, so - `state.searchResults` *is* what the pane serves), never by branching in each painter. -- The nearest list is unreachable from the human surface for that query. That cost is accepted - because it is bounded to **one query**, never a whole vault — the distinction from ADR-0014. -- Exit-gate rows: zero labelled negatives served, zero labelled positives cut, zero dense-fixture - title-queries cut — structural zeros, no headroom. Discovery rows are unchanged, because search's - rule moves no discovery rank; movement there would be a bug. -- The per-hit **tail** fold is unshipped — since GH #206 (2026-08-25) by **measurement** rather - than by missing labels: `tail_relevant` deepened the labels to per-hit exhaustiveness, a - four-family prefix-cut bake-off ran on both corpora plus a real vault, and every admissible - family is vacuous (2–23 of the 367 rows an oracle fold reaches), because the fused order is not - an evidence order and D1 rightly forbids a fold from re-sorting it. The tail complaint is an - ordering problem — standing evidence for the reranker seam, not for a disclosure rule. The - bake-off re-derives this verdict every `make eval` run. diff --git a/ADRs/0016-rendering-note-content-is-a-trust-boundary.md b/ADRs/0016-rendering-note-content-is-a-trust-boundary.md deleted file mode 100644 index aae915b..0000000 --- a/ADRs/0016-rendering-note-content-is-a-trust-boundary.md +++ /dev/null @@ -1,26 +0,0 @@ -# ADR-0016 — Rendering note content is a trust boundary - -- **Status:** Accepted · 2026-07-26 -- **Refs:** invariants E5 · GH #77 · `ui/src/sanitize.ts`, `ui/src/links.ts` - -## Context - -Authorship is not trust. A `.md` can come from a shared vault, a download, or a web clip — and model -output is the same class of input. The webview *is* the application, so injected markup or a hostile -link scheme is not a rendering bug, it is code execution. - -## Decision - -- **One** Markdown→HTML seam (`renderMarkdown`), which sanitizes its output — DOMPurify wired as - `marked`'s `postprocess` hook, so every call site is covered *by construction*. -- Every value B2 itself interpolates into chrome goes through `escapeHtml`. -- The webview CSP (`default-src 'self'`, no inline scripts) is a **second, independent layer** — never - the sole guard. -- A note's link **never navigates the webview**: `http`/`https`/`mailto` are an OS handoff performed - host-side behind a scheme allow-list the host re-checks; every other scheme is **refused**, not - handed to whatever app registered it. - -## Consequences - -- Any new render path must go through the one seam, or it is a hole. -- Model output is rendered under exactly the same rules as note content. diff --git a/ADRs/0017-the-gui-is-keyboard-complete-with-one-chord-registry.md b/ADRs/0017-the-gui-is-keyboard-complete-with-one-chord-registry.md deleted file mode 100644 index c67d818..0000000 --- a/ADRs/0017-the-gui-is-keyboard-complete-with-one-chord-registry.md +++ /dev/null @@ -1,35 +0,0 @@ -# ADR-0017 — The GUI is keyboard-complete, and every chord lives in one registry - -- **Status:** Accepted · 2026-07-28 (menu bar declared 2026-07; rebinding 2026-07-28) -- **Refs:** invariants K1 · GH #78, #119, #121 · `ui/src/bindings.ts` - -## Context - -Keyboard completeness decays silently: one new pane that only responds to a click, and the promise is -gone. And a chord table that exists in three places (dispatcher, editor keymap, help sheet) drifts. - -## Decision - -- Every action the mouse can take has a keyboard path. Panes follow ARIA patterns over the **same row - order the painter uses**, so the arrows and the eye cannot disagree; every pane restores focus - across its own repaint, and every overlay traps and restores it. -- **Chords are declared once** in `ui/src/bindings.ts`; the dispatcher, the editor keymap, and the - help sheet all derive from it. Arrow families are in the registry too — navigation modules own - *command → move*, the registry owns *key → command*. -- Four checkers gate the table: same-scope conflicts, CodeMirror's ~100 stock bindings, and the - **macOS menu bar's** accelerators — which is why the menu bar is *declared* in - `b2-desktop/src/menu.rs` rather than inherited from Tauri's default, since AppKit dispatches a menu - accelerator before the webview sees the key at all. -- **The chords are the user's**: every row is re-recordable, stored as a UI preference in - `localStorage` (never vault state). A candidate is judged against the table it *would* produce — - refused on a same-scope or menu-bar clash, merely warned for a shadow or CodeMirror overlap — and - the recorder reads the one thing no table can know: a chord producing **no keydown** was taken - upstream by macOS. -- The loader re-judges what it reads, so a hand-edited store cannot install a keyboard you cannot use - to fix itself. - -## Consequences - -- A new surface owes the four obligations listed in `crates/b2-desktop/CLAUDE.md`. -- A chord live in the app is B2's to document, whoever authored it — an unenumerated chord cannot be - found, and the app cannot warn about landing on it. diff --git a/ADRs/0018-ci-runs-make-ci-and-nothing-else.md b/ADRs/0018-ci-runs-make-ci-and-nothing-else.md deleted file mode 100644 index 865e74f..0000000 --- a/ADRs/0018-ci-runs-make-ci-and-nothing-else.md +++ /dev/null @@ -1,37 +0,0 @@ -# ADR-0018 — CI runs `make ci` and nothing else - -- **Status:** Accepted · 2026-07-25 · migrated `just` → `make` 2026-08-25 -- **Refs:** GH #87 · `Makefile`, `.github/workflows/ci.yml` - -## Context - -A check re-specified in workflow YAML is a check that can drift from the one you run locally. - -## Decision - -- **Every check lives in a `make` target.** The workflow supplies only the environment - (toolchain, Node, the Rust cache) and runs `make ci`. Two gates: `make check` (fast, the - working loop) and `make ci` (complete, what CI runs — run it before pushing). -- CI runs on **macos-latest**: B2 ships on macOS only, and the Xcode CLT the WebView links - against — and `make` itself — are preinstalled, so no separate task-runner install step exists. -- CI is model-free by construction (`b2-core` never links candle; CLI tests spawn the binary under - `B2_EMBEDDER=fake`). -- One CI-only step follows the gate: **`git diff --exit-code`**, asserting no stage edited a tracked - file — so the lockfile cannot drift from `package.json` and no suite can mutate a committed fixture. -- **Advisory-but-exit-0 output is a hole in a gate.** Clippy warns and exits 0 (hence `-D warnings` - everywhere); `npm install` reports vulnerabilities and exits 0 (hence a separate `make audit`). - When wiring a new tool into a recipe, check its exit code on a *failing* input. - -## Consequences - -- Targets carry a trailing `## ` comment and a `##@ ` section marker: `make help` - (the no-args default) parses these into a grouped command reference — the Make analogue of the - `just` recipe attributes this ADR originally described, which `just` itself rendered the same - way from `[group('…')]` + `[doc("…")]`. -- `make audit` is deliberately **not** a prerequisite of `ui-install` or `check` — a fresh advisory - must not stop the app launching. -- **2026-08-25: migrated the task runner from `just` to `make`**, removing the `just` dependency. - The decision this ADR records — one enforcement point, `check`/`ci` as the two gates, no - advisory-but-exit-0 holes — is unchanged; only the tool executing it is. `justfile` → `Makefile`, - `extractions/setup-just@v4` dropped from CI (nothing left to install), and every `just ` - reference across the docs became `make `. diff --git a/ADRs/0019-build-our-own-sqlite-index-engine.md b/ADRs/0019-build-our-own-sqlite-index-engine.md deleted file mode 100644 index b951433..0000000 --- a/ADRs/0019-build-our-own-sqlite-index-engine.md +++ /dev/null @@ -1,43 +0,0 @@ -# ADR-0019 — Build our own SQLite index engine; qmd is a reference, not a dependency - -- **Status:** Accepted · 2026-06-29 -- **Refs:** `index-engine.md` §1–§3 · [tobi/qmd](https://github.com/tobi/qmd) - -## Context - -[qmd](https://github.com/tobi/qmd) is an excellent local Markdown search engine — SQLite + FTS5 + -`sqlite-vec`, ~900-token Markdown-aware chunks, BM25 ⊕ vector ⊕ RRF ⊕ LLM rerank, local GGUF models, -MIT-licensed. It proves the pipeline runs on-device. The question was whether to depend on it. - -## Decision - -**Build the index engine; take qmd as a design reference.** The disagreement is **scope, not -quality**: qmd is a *search engine*, and B2 is a *typed graph with hybrid retrieval over it*. qmd -models no typed edges, no backlinks, and nothing that rewrites links on a move — which are the -reasons B2 exists. Wrapping it would mean a second store for everything that makes B2 B2, and two -sources of truth to reconcile; the retrieval glue actually wanted is ~300 lines. - -- **Borrowed wholesale:** the chunking heuristic, the RRF formula and `k = 60`, the position-aware - blend, the asymmetric query/document prompt discipline, the JSON/`--explain` agent-output - discipline, and the MCP surface idea. -- **Discarded:** the npm/Node packaging and the "the DB is the product" framing. -- **SQLite is the substrate** because it gives one embedded store for every *queryable* concern at - once — full-text (FTS5), vectors (plain tables, ADR-0006), and the typed graph — transactionally, - so discovery's candidate generation joins all three in one query. That single-store property is - worth more than anything inheritable from qmd. -- Because the engine **does** provide vector search, the engine-gated call resolves in favour of - **semantic search in v1** — exact, in-process, no ANN. -- **The single-binary goal picked the language, not the engine.** SQLite and FTS5 are - language-agnostic; B2 is a Rust workspace (`rusqlite` with bundled SQLite, `candle` for the - embedder) because a Node runtime is the least aligned with shipping one binary — another reason - not to inherit qmd's runtime by depending on it. - -## Consequences - -- The one genuinely hard part is **not the engine** — it is producing embeddings inside a single - binary, which is orthogonal to choosing SQLite (ADR-0020). -- A reranker is a clean fast-follow behind a post-fusion seam, not a redesign; retrieval quality is - good without it. -- Net: qmd answers "can great hybrid search run locally over Markdown?". B2's question is one layer - up — "can that retrieval live inside a typed, traversable, agent-operated graph I fully own, in a - single binary?" diff --git a/ADRs/0020-embeddings-inside-the-single-binary.md b/ADRs/0020-embeddings-inside-the-single-binary.md deleted file mode 100644 index 3e8632b..0000000 --- a/ADRs/0020-embeddings-inside-the-single-binary.md +++ /dev/null @@ -1,31 +0,0 @@ -# ADR-0020 — Embeddings inside the single binary: candle + hf-hub, provisioned by `b2 init` - -- **Status:** Accepted · 2026-06-30 (built 2026-07-01) -- **Refs:** invariants M1, M2 · `index-engine.md` §6 · GH #40 - -## Context - -Semantic search needs vectors from somewhere, and this is the one place the architecture meets real -friction — independent of the store (ADR-0019). qmd's answer (`node-llama-cpp` + auto-downloaded -GGUF + a JS runtime) tensions B2's single-binary, no-install-ritual goal directly. - -## Decision - -- **Runtime: `candle` + `hf-hub`** — pure-Rust inference compiled into the binary, no external ONNX - Runtime to ship, with `hf-hub` as the download seam. -- **Not bundled, and never a surprise download.** An explicit **`b2 init`** downloads and verifies the - model into a shared XDG cache; `reindex`/`search` fail fast with "run `b2 init`" if it is absent. -- **The model source is configurable** (default an HF repo id; overridable to a mirror, another repo, - or a local path for offline installs) via `$XDG_CONFIG_HOME/b2/config.toml`. -- **Default model `BAAI/bge-base-en-v1.5`** (BERT-family, 768-dim, ungated), CLS-pooled and - L2-normalized, with bge's asymmetric query prefix. **EmbeddingGemma-300M was the first choice and - lost on friction**: it is *gated* on Hugging Face (401 without a token + license acceptance), which - defeats a friction-free `b2 init`. It stays selectable for anyone who provides a token. -- The dimension is read authoritatively from the model's own `config.json` (`hidden_size`), so - configuration cannot lie about it. - -## Consequences - -- All heavy ML deps stay in `b2-embed` alone; `b2-core` never links candle (ADR-0005), and the fake - embedder is the CI default, so model quality never enters the fast suite. -- The recorded embedding-space identity and the device-swap rules that follow are ADR-0007. diff --git a/ADRs/0021-concurrency-is-serialized-on-sqlites-own-locks.md b/ADRs/0021-concurrency-is-serialized-on-sqlites-own-locks.md deleted file mode 100644 index e6c13c3..0000000 --- a/ADRs/0021-concurrency-is-serialized-on-sqlites-own-locks.md +++ /dev/null @@ -1,49 +0,0 @@ -# ADR-0021 — Many readers, one writer, serialized on SQLite's own locks - -- **Status:** Accepted · 2026-08-23 -- **Refs:** invariants C1 · `index-engine.md` §3 · GH #55, #111, #114 - -## Context - -A personal vault is opened by more than one process at a time — the desktop app, a `b2` command, a -backgrounded `b2 reindex &` — and the index is a disposable projection (ADR-0002), so a reader must -never be refused because a writer is rebuilding one. Three defects made that untrue, each measured: - -| Issue | The race | Measured | -|---|---|---| -| [#111](https://github.com/AlteredCraft/B2/issues/111) | `journal_mode = WAL` is the one statement in `open` that takes a write lock, and `busy_timeout` does not cover it — SQLite consults the busy handler only for a write lock taken from no transaction at all | ~4% of racing opens took an immediate `SQLITE_BUSY`; the documented `b2 reindex & ; b2 status` flow failed ~40% of the time, usually killing the reindex | -| [#114](https://github.com/AlteredCraft/B2/issues/114) | `migrate` was read-then-decide-then-write over ~30 separately-committed DDL statements, so two openers both rebuilt and interleaved | over 320 racing opens: 10 failed outright, and 8 missing-table observations were left behind by rounds where *every* opener returned `Ok` | -| #114 (again) | `ensure_embedding_space` is the same drop-and-rebuild, and the `#55` advisory lock is the CLI's alone — a desktop reindex and a `b2 reindex` genuinely overlap | 70 of 80 workers errored and **every** round lost vectors | - -`busy_timeout` never applied to the second and third: nothing contended for a lock, the statements -all succeeded in the wrong order. - -## Decision - -- **Serialization is SQLite's own write lock** (`BEGIN IMMEDIATE` + a bounded retry), not a second - mechanism layered over the database. Both drop-and-rebuilds — the schema migration and the vector - tables — check, then re-check *inside* the lock, then rebuild once, in **one transaction** (DDL is - transactional in SQLite), so nothing observes a half-built shape. -- **An advisory lock file was rejected.** It would be a third concurrency mechanism guarding state - the database already guards, and the weaker guard exactly where it matters: a vault on a network - share or synced folder is where `flock` stops meaning anything. -- **The fast path takes no write lock at all.** An index already at the current `schema_version` - costs two reads, which is what makes C1's "a reader is never refused" true rather than hoped for. -- **"Complete" is checked, not assumed:** a current stamp over missing tables is rebuilt from empty, - because surviving rows would look up-to-date to an incremental reindex and break S3. -- **The WAL flip retries in `b2-core`**, since it is the one statement `busy_timeout` cannot cover; - its budget is therefore large where the DDL rebuilds' is small (theirs sits *on top of* the - timeout). The flip's result is **read back**, because a filesystem with no shared-memory support - declines it with `SQLITE_OK` and the old mode — a decline is said out loud, never retried. -- **Concurrent writers stay single-in-flight** by the `reindex` advisory lock (the CLI) and the - single-in-flight embed slot (the desktop). Readers never take either, which is precisely why - neither can cover the rebuilds above. - -## Consequences - -- Waiting is bounded, deliberately: past the budget a stuck writer is *reported* rather than hung on. -- The races are covered by probes, not proofs, and the tests say so — `tests/substrate.rs` holds the - contended lock outright for the deterministic gate and keeps the thread-race version beside it, - labelled as the smoke test it is (green ~88% of the time against the unfixed `open`). -- Every new drop-and-rebuild in `db.rs` inherits this shape. Adding one without it reintroduces the - same defect, which is the quiet kind: an index left incomplete while every opener returned `Ok`. diff --git a/ADRs/0022-the-chat-seam-carries-tool-calls-for-bounded-turns.md b/ADRs/0022-the-chat-seam-carries-tool-calls-for-bounded-turns.md deleted file mode 100644 index fd79bc1..0000000 --- a/ADRs/0022-the-chat-seam-carries-tool-calls-for-bounded-turns.md +++ /dev/null @@ -1,57 +0,0 @@ -# ADR-0022 — The chat seam carries tool calls; a tool-using turn is bounded and degrades - -- **Status:** Accepted · 2026-09-20 -- **Refs:** invariants M1, E2, E5, S4 · ADR-0005, ADR-0011, ADR-0012 · GH #210 - -## Context - -Grounded chat (flow ④) is one-shot: B2 retrieves, the model answers. "Why was this note suggested?" -is a different question — its evidence is not a search result but the output of B2's own reads -(`similar`'s matched passages, the graph, the notes themselves), and which of them matter depends on -what the first ones show. The requirement was that the chat model reach that evidence **through B2 -tools**, not be handed a fixed bundle. - -GH #210 argues against a loop inside `b2-core`, on two grounds: small local models execute tool loops -worst, and a loop in the caller keeps the engine dumb. Both were measured here rather than assumed. -On Ollama, `llama3.2:3b` and `gemma4` both emit well-formed tool calls for these tools. And a model -handed a lookup it did not make stops looking: with the pair evidence pre-seeded into the -conversation, neither model called anything further in any run. - -## Decision - -- **`LlmProvider` carries tool calls.** `ChatRequest` gains `tools` (name, description, JSON-Schema - parameters) and `exchanges` (each call with B2's result, replayed after the turns); `Completion` - gains `tool_calls`; `RequestKind::Agent` marks the turn. Still one trait method, still sync - (ADR-0011). A request with no tools is byte-for-byte the request it was, so `ask` is unchanged. -- **A tool is one read on the `Vault` façade**, run by the façade for the model. No tool writes (W1), - embeds a query, or reaches outside the vault. Arguments are model output, so they are untrusted: a - malformed or unknown call gets an `error:` *result*, never a failed turn. -- **The loop lives in `b2-core`, and it is bounded.** At most `MAX_TOOL_ROUNDS` model calls and - `MAX_CALLS_PER_ROUND` tool calls each; the last round offers no tools. The loop is in the core, not - an adapter, for ADR-0012's reason: two adapters running their own loops would be two behaviours. -- **The model makes the lookups; B2 guarantees the grounding.** Round 1's text is never streamed. A - model that calls nothing there, or a provider that refuses tools, degrades to the same evidence in - one plain grounded request. A model that calls tools but skips the pair lookup has it appended, - marked `seeded`. -- **One reply's tool calls are capped, and the cap fails loudly.** `LlmConfig::max_tool_calls` - (`B2_LLM_MAX_TOOL_CALLS`, default 64) bounds the calls — and the highest call `index` — a reply may - carry, because the stream's byte cap does not bound the table a sparse `index` can make of those - bytes. Past it the call fails with a typed error (`Error::ToolCallLimit`) that the degrade above - must not swallow: a reply this far outside the protocol is a broken or hostile server, not a model - without tool support. -- **Passages are numbered on one ledger per turn**, across every tool result, and `[n]` resolves - against it — the citation contract of flow ④, unchanged. -- **The answer says which tools ran** (`AnswerView.tools`, `seeded` distinguishing B2's own call), so - a surface never overstates what the model did. - -## Consequences - -- `Vault::why_similar` is the one tool-using turn. `ask` stays one-shot until #210's experiment says - otherwise; nothing here prejudges it, and its loop-in-the-caller shape (an MCP or `serve` adapter) - remains open — these tool definitions are what it would expose. -- A cancel during the hidden lookup round lands at the first visible token. Accepted: the round is - short, and the alternative is showing text the turn may discard. -- The degrade costs one refused request against a model with no tool support, on every why-turn. -- `FakeLlm` scripts an agent turn off the request's *structure* (call every tool that needs no - arguments once, then answer), so the engine, CLI and desktop suites cover the loop model-free (E2). - How well a given model uses the tools is a quality question, which is ADR-0013's harness. diff --git a/ADRs/README.md b/ADRs/README.md deleted file mode 100644 index b871784..0000000 --- a/ADRs/README.md +++ /dev/null @@ -1,36 +0,0 @@ -# Architecture Decision Records - -One file per **key architectural choice** — the decisions that would be expensive to reverse and -whose *why* is not readable off the code. Not a changelog: features, bug fixes and tuning do not get -an ADR. Feature history lives in [GitHub Issues](https://github.com/AlteredCraft/B2/issues) and git. - -**Keep them terse** — Context / Decision / Consequences, a page at most. When a decision is -overturned, mark the old ADR `Superseded by ADR-NNNN` and write a new one; never edit history. - -Normative detail lives in `docs/invariants.md` (the register — it wins on conflict); these -records say *why* an entry reads the way it does. - -| # | Decision | -|---|---| -| [0001](0001-design-docs-are-normative.md) | Design docs are normative; the code is a projection of them | -| [0002](0002-vault-is-truth-index-is-a-disposable-projection.md) | The vault is the source of truth; the index is a disposable projection | -| [0003](0003-note-identity-is-the-vault-relative-path.md) | A note's identity is its vault-relative path | -| [0004](0004-markdown-is-the-only-surface-b2-writes.md) | Markdown is the only surface B2 writes, and B2 makes no unbidden writes | -| [0005](0005-ai-behind-enumerated-seams.md) | AI lives behind two enumerated seams; the core is model-free | -| [0006](0006-vectors-in-plain-tables-content-addressed.md) | Vectors live in plain SQLite tables, scored in-process, content-addressed | -| [0007](0007-embedding-space-identity-includes-the-device.md) | The embedding space has one recorded identity, and the device is part of it | -| [0008](0008-hybrid-retrieval-bm25-plus-vector-fused-by-rrf.md) | Retrieval is BM25 ⊕ vector KNN, fused by RRF | -| [0009](0009-discovery-is-model-free-at-surface-time.md) | Discovery surfaces candidates; the human commits the link | -| [0010](0010-typed-graph-two-homes-closed-vocabulary.md) | The typed graph: two authored homes, frontmatter-wins, a closed verb core | -| [0011](0011-no-async-runtime-in-b2-code.md) | No B2 code is async; no B2 crate starts a runtime | -| [0012](0012-one-typed-facade-two-dumb-adapters.md) | One typed façade, two dumb adapters | -| [0013](0013-model-quality-is-measured-out-of-ci.md) | Model quality is measured out of CI, by a labelled harness | -| [0014](0014-discovery-always-serves-the-ranked-prefix.md) | Discovery always serves the ranked top-N; no anchor-local existence gate | -| [0015](0015-a-served-search-result-is-a-claim-of-evidence.md) | A served search result is a claim of evidence | -| [0016](0016-rendering-note-content-is-a-trust-boundary.md) | Rendering note content is a trust boundary | -| [0017](0017-the-gui-is-keyboard-complete-with-one-chord-registry.md) | The GUI is keyboard-complete, and every chord lives in one registry | -| [0018](0018-ci-runs-make-ci-and-nothing-else.md) | CI runs `make ci` and nothing else | -| [0019](0019-build-our-own-sqlite-index-engine.md) | Build our own SQLite index engine; qmd is a reference, not a dependency | -| [0020](0020-embeddings-inside-the-single-binary.md) | Embeddings inside the single binary: candle + hf-hub, provisioned by `b2 init` | -| [0021](0021-concurrency-is-serialized-on-sqlites-own-locks.md) | Many readers, one writer, serialized on SQLite's own locks | -| [0022](0022-the-chat-seam-carries-tool-calls-for-bounded-turns.md) | The chat seam carries tool calls; a tool-using turn is bounded and degrades | diff --git a/CLAUDE.md b/CLAUDE.md index 0bc436d..c251c12 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -5,156 +5,141 @@ Guidance for Claude Code (claude.ai/code) and other agents working in this repos ## What B2 is for -B2 is my daily notes app, built to replace Obsidian. The reason to replace it is research: -Obsidian is a good place to keep notes and a poor place to think with them. B2 keeps what -Obsidian gets right (plain Markdown in a folder I own, usable by any editor, no lock-in) and -adds an intelligence layer that helps me do research across what I have already written: - -- **Find the connections I haven't made.** Every note shows the related notes it isn't linked - to yet, and **Why?** explains what the two have in common, with citations. -- **Say how ideas relate, not just that they do.** A link can be typed as `supports` or - `contradicts`, because stance is the one thing similarity can't infer. -- **Get answers I can check.** Search says "no matches" when the vault holds no evidence, and - chat answers only from my notes, citing each claim back to the passage it came from. -- **Let agents work the vault too.** Every command has a `--json` form, so an agent can search, - read and walk the graph the way I do. - -Two ideas shape every decision, and they are the ones to protect: +B2 is my daily notes app, built to replace Obsidian. Obsidian is a good place to keep notes +and a poor place to think with them. B2 keeps what it gets right (plain Markdown in a folder +I own, usable by any editor) and adds an intelligence layer for research across what I have +already written: + +- **Find the connections I haven't made.** Every note shows the related notes it isn't + linked to yet, and **Why?** explains what the two have in common, with citations. +- **Say how ideas relate.** A link can be typed `supports` or `contradicts`, because stance + is the one thing similarity can't infer. +- **Get answers I can check.** Search says "no matches" when the vault holds no evidence, + and chat answers only from my notes, citing each claim. +- **Let agents work the vault too.** Every command has a `--json` form. + +Two ideas shape every decision. Protect them: 1. **The vault is the truth; the index is disposable.** `index = projection of (the vault - directory)`. Drop `.b2/b2.sqlite`, rebuild, get the same index. So I can move, split, merge - and rewrite notes without fear, and B2 writes nothing I didn't ask for. -2. **Build for tomorrow's model.** Every AI part sits behind a small seam, and B2 does the least - today's model needs. A better model should drop in, not force a redesign. + directory)`. Drop `.b2/b2.sqlite`, rebuild, get the same index. +2. **Build for tomorrow's model.** Every AI part sits behind a small seam and does the least + today's model needs, so a better model drops in without a redesign. -The human is the judge. B2 suggests; I decide what gets linked, typed and kept. +B2 suggests; I decide what gets linked, typed and kept. -## Where we are, and how work starts +## How work starts -The build-out is done: the engine, the `b2` CLI, the desktop app and grounded chat all work end -to end. The next phase is **using B2 every day and letting that use drive the work.** +The build-out is done. The work now is using B2 every day and fixing what that shows. -- New work starts from an issue labelled **`observed`**: what I was doing, what I expected, what - happened, how often. Prefer fixing friction I actually hit over tuning I might need. -- The backlog was reset on 2026-09-23. Closed ideas (rerankers, scaling levers, resource search, - sync, highlights) keep their write-ups. Reopen one only when an observation calls for it. -- Measure before you tune. A change to ranking or model quality is judged by the eval harness +- New work starts from an issue labelled **`observed`**: what I was doing, what I expected, + what happened, how often. Prefer fixing friction I hit over tuning I might need. +- The backlog was reset on 2026-09-23. Closed ideas (rerankers, scaling, resource search, + sync, highlights) stay closed until an observation calls for one. +- Measure before you tune. Ranking and model quality are judged by the eval harness ([docs/evals.md](docs/evals.md)), never by intuition. -- If a request would pull B2 away from the vision above (a proprietary format, an unbidden write, - an AI step with no seam), say so before building it. +- If a request would pull B2 away from the two ideas above (a proprietary format, an + unbidden write, an AI step with no seam), say so before building it. -## Where the truth lives +## The docs -The code is a projection of the spec, and comments cite it (`data-model.md §2`, invariant ids -like `S2`, `D1`). Read the relevant part before changing behaviour. - -| Source | Role | +| Doc | Holds | |---|---| -| [docs/invariants.md](docs/invariants.md) | What must always be true, cited by id. **On conflict with anything else, it wins.** | -| [docs/data-model.md](docs/data-model.md) | The *what*: notes, connections, the two storage tiers, the relation vocabulary. | -| [docs/index-engine.md](docs/index-engine.md) | The *how*: the SQLite projection, its tables, the four flows, the Why? tool turn. | -| [docs/architecture.md](docs/architecture.md) | The map: crates, flows, seams, how the tests hold it up. Start here in the code. | -| [docs/quickstart.md](docs/quickstart.md) | Using B2: commands, config, and **every environment variable**. | -| [ADRs/](ADRs/README.md) | Why the above reads the way it does. Add one only for a decision that is expensive to reverse and whose *why* isn't readable off the code. | -| [GitHub Issues](https://github.com/AlteredCraft/B2/issues) | The backlog. Decision history is the issue that drove a change plus the commit that shipped it. | - -`crates/b2-desktop/CLAUDE.md` holds the desktop crate's own rules. Read it before touching that -crate or `ui/`. +| [docs/invariants.md](docs/invariants.md) | What must always be true, cited by id (`S2`, `D1`). **It wins any conflict.** Read the entries for the area you are changing. | +| [docs/decisions.md](docs/decisions.md) | The few key decisions and why, cited by id (`ADR-0014`). | +| [docs/data-model.md](docs/data-model.md) | What a vault holds: notes, links, the verbs, resources. | +| [docs/architecture.md](docs/architecture.md) | How it's built: the crates, the index, the flows. | +| [docs/evals.md](docs/evals.md) | Measuring search, discovery and chat quality. | +| [docs/quickstart.md](docs/quickstart.md) | Using B2: commands, config, every environment variable. | +| [crates/b2-desktop/CLAUDE.md](crates/b2-desktop/CLAUDE.md) | Rules for the desktop crate and `ui/`. Read it before touching either. | + +Keep them short and current: + +- Each fact has one home. When behaviour changes, update that home in the same change. +- Docs describe how B2 works now. History lives in GitHub Issues and git. +- Code comments cite an invariant id, a decision id, or a doc by name, never a section + number. +- Add a decision only when it is expensive to reverse and its reason can't be read off the + code. ## The shape of the system -A Cargo workspace plus a frontend. Details and the reasons live in -[docs/architecture.md](docs/architecture.md). - -- **`b2-core`**: the whole engine, and the one typed API, the `Vault` façade (`vault.rs`). - Model-free, deterministic, fast to test. Never add candle, tokenizers or an HTTP client here. +- **`b2-core`**: the whole engine and the one typed API, the `Vault` façade (`vault.rs`). + Model-free and deterministic. Never add candle, tokenizers or an HTTP client here. - **`b2-embed`**: the real local embedder (candle, bge). All heavy ML dependencies live here. -- **`b2-llm`**: the real chat provider, a small sync OpenAI-compatible streaming client. Ollama - is the guided default; any compatible endpoint works. -- **`b2-cli`** and **`b2-desktop`** (Tauri) are the two **dumb adapters**: parse input, make one - `Vault` call, render the result. The desktop reuses the CLI's `--json` view types as its IPC - contract. The frontend is `ui/` (Vite, vanilla TypeScript, CodeMirror 6). - -Engine logic belongs in `b2-core`, behind the façade, never in an adapter. Add a façade operation -when a command needs it; don't pre-build surface. - -## Principles that hold everywhere - -- **No unbidden writes.** Reading, walking and reindexing write nothing to the vault. Every write - is the mechanics of a command I ran, and the body of a note is always mine (W1 to W3). -- **A note is its path.** Identity is the vault-relative path; nothing is minted or stamped into - files. -- **Degrade honestly.** An unembedded vault still searches by keyword; a model that can't use - tools still gets an answer; a failed condense falls back to the raw question. Say what is - missing rather than failing or pretending. -- **Model output is untrusted.** Chat stores nothing, never rewrites its answer, and a bad tool - call gets an error result, not a crash. Rendering note content is a trust boundary (ADR-0016). -- **Limits are configurable and loud.** A cap gets a clear error, a setting, and a place in - Settings, not a silent trim. -- **Determinism in the core.** No wall clock and no randomness in `b2-core`: time is passed in, - and logging is only emitted there (the adapters own the subscriber and the clock). -- **Errors shown to people are generic and actionable.** Nothing internal (sqlite, io, serde) - leaks; `B2_DEBUG` opts into detail. -- **No async.** Nothing in B2's code is async, and the `b2` binary links no tokio (ADR-0011). +- **`b2-llm`**: the real chat provider, a small sync OpenAI-compatible client. Ollama by + default. +- **`b2-cli`** and **`b2-desktop`** (Tauri, frontend in `ui/`): two **dumb adapters**. Parse + input, make one `Vault` call, render the result. The desktop reuses the CLI's `--json` + view types as its IPC contract. + +Engine logic belongs in `b2-core`, behind the façade, never in an adapter. Add a façade +operation when a command needs it; don't pre-build surface. + +## Principles + +- **No unbidden writes.** Reading and reindexing write nothing to the vault. Every write is + the mechanics of a command I ran, and a note's body is always mine (W1 to W3). +- **A note is its path.** Nothing is minted or stamped into files (L1). +- **Degrade honestly.** An unembedded vault still searches by keyword; a model that can't + use tools still gets an answer. Say what is missing instead of failing or pretending. +- **Model output is untrusted.** Chat stores nothing and never rewrites its answer, a bad + tool call gets an error result, and rendering is a trust boundary (E5). +- **Limits are configurable and loud.** A cap gets a clear error and a setting, not a + silent trim. +- **The core is deterministic.** No wall clock or randomness in `b2-core`. Time is passed + in; the adapters own the clock and the log subscriber (E1). +- **Errors shown to people are generic and actionable.** Nothing internal leaks; `B2_DEBUG` + opts into detail (E4). +- **No async.** Nothing in B2 is async, and the `b2` binary links no tokio (ADR-0011). ## Working in the repo -`make` with no arguments lists every target. Two gates, the same ones CI runs (ADR-0018): +`make` lists every target. CI runs `make ci` and nothing else, so every check lives in a +make target. ```bash make check # fast loop: fmt, clippy -D warnings, engine suite, ui/ suite make ci # everything CI runs; run it before pushing ``` -Things the target list won't tell you: - -- **Offline by default.** `B2_EMBEDDER=fake` and `B2_LLM=fake` give deterministic stand-ins; - the test suite runs on them. The real model needs `make init` once; real chat needs +- **Offline by default.** `B2_EMBEDDER=fake` and `B2_LLM=fake` are deterministic stand-ins, + and the suite runs on them. The real model needs `make init`; real chat needs `ollama serve`. -- **Commands that write need an explicit vault** (`-C`, a path, or `B2_VAULT_PATH`), so a typo - can't touch the wrong folder. -- **Metal is a build switch, not a runtime one.** `make app` picks it on Apple Silicon. The - device is part of the embedding space's identity, so switching re-embeds the vault (ADR-0007). -- **Whole-workspace `cargo test`** embeds `ui/dist`, so run `make ui-build` once first. +- **Commands that write need an explicit vault** (`-C`, a path, or `B2_VAULT_PATH`). +- **Metal is a build switch.** `make app` picks it on Apple Silicon, and switching + re-embeds the vault (M2). +- **Whole-workspace `cargo test`** embeds `ui/dist`, so run `make ui-build` first. - **Coverage** needs `cargo-llvm-cov` and `llvm-tools-preview`; `make doctor` checks both. +- **A new tool in a make target** must exit non-zero on a failing input. Check that it + does. ### Tests -- **Keep `cargo test` fast, deterministic and model-free.** Anything that needs the real model - belongs in the eval harness, where it actually runs. -- **Never `#[ignore]` a test.** A test that is hard to write is a signal: is it testing a real - invariant or an implementation detail? Is the fault in the test or the system? If the answer - isn't clear, ask me rather than weakening the assertion. +- **Keep `cargo test` fast, deterministic and model-free.** Anything that needs the real + model belongs in the eval harness. +- **Never `#[ignore]` a test.** A test that is hard to write is a signal: is it testing an + invariant or an implementation detail? Is the fault in the test or the system? If that + isn't clear, ask me instead of weakening the assertion. - **A test's name is part of its contract.** It must not claim more than the body asserts. -- Integration tests copy `fixtures/golden-vault/` into a temp dir first, so no suite can change - the repo's fixtures. Shared helpers live in `crates/b2-core/tests/common/mod.rs`; the one - deliberate copy is the tracing `Capture` writer in `logging.rs` and - `discover_query_count.rs`, which need their own test binaries. (`fixtures/test-vault/` is a - larger synthetic vault for out-of-CI experiments.) - -## Idiomatic Rust +- Integration tests copy `fixtures/golden-vault/` into a temp dir first. Shared helpers + live in `crates/b2-core/tests/common/mod.rs`; the one deliberate copy is the tracing + `Capture` writer in `logging.rs` and `discover_query_count.rs`, which need their own + binaries. -### Data modeling +## Rust -- Ownership forms a tree or DAG, never a cycle. One clear owner per value. -- For references between values, use keys (`slotmap`, or `Vec` indices if nothing is removed). - `Rc>` and `Arc>` are last resorts. -- Prefer owned fields to borrowed ones. A lifetime on a struct usually wants owned data or a key. - The exception is a short-lived, `Copy`, read-only view passed into one call (like `NoteRow` in +- **Ownership is a tree, never a cycle.** Refer between values by key or index. + `Rc>` and `Arc>` are last resorts. No self-referential structs. +- **Prefer owned fields.** A lifetime on a struct usually wants owned data or a key. The + exception is a short-lived read-only view passed into one call (like `NoteRow` in `db.rs`); say so in its doc comment. -- Never silence the borrow checker with a reflexive `.clone()`. Ask who owns the value, and - whether the relationship should be an ID instead of a pointer. -- No self-referential structs; restructure with indices. -- No `.unwrap()` or `.expect()` in production paths, even for an invariant you believe can't - fail. Degrade gracefully instead of panicking. - -### Style and structure - -- Errors: `thiserror` enums wherever variants are matched, including the CLI's `CliError`. - `anyhow` only where errors are just propagated and printed. Never hand-roll `From`/`Display`. -- Accept `&str` and `&[T]`; return owned types. -- Prefer iterator chains to index loops. +- **Never silence the borrow checker with a reflexive `.clone()`.** Ask who owns the value. +- **No `.unwrap()` or `.expect()` in production paths.** Degrade instead of panicking. +- **Errors:** `thiserror` enums wherever variants are matched, including `CliError`. + `anyhow` only where errors are just propagated and printed. Never hand-roll `From` or + `Display`. +- Accept `&str` and `&[T]`; return owned types. Prefer iterator chains to index loops. - No `async`, generics, traits or macros until there is a concrete need. -- `unsafe` needs a `// SAFETY:` comment stating why it is sound. -- Derive `Debug` on public data types (`Clone`/`PartialEq` where sensible). -- Keep modules small and named for their domain. Doc comments state intent, not mechanics. +- `unsafe` needs a `// SAFETY:` comment saying why it is sound. +- Derive `Debug` on public data types (`Clone` and `PartialEq` where sensible). +- Small modules named for their domain. Doc comments state intent, not mechanics. diff --git a/README.md b/README.md index 5e7590a..b19ca0a 100644 --- a/README.md +++ b/README.md @@ -14,52 +14,41 @@ common, how they bear on each other, and what they actually say. ## What it does - **Shows the connections you haven't made.** Beside every note is a ranked list of related - notes it isn't linked to yet. Click **Why?** on one and a model explains what the two have - in common, citing the passages, using B2's own read-only tools to look things up. -- **Lets you say how ideas relate.** Link two notes with one click and type the link: - `references`, `supports` or `contradicts`. Similarity can tell that two notes are about the - same thing; only you know whether one backs the other up or argues against it. + notes it isn't linked to yet. Click **Why?** and a model explains what the two have in + common, citing the passages. +- **Lets you say how ideas relate.** Link two notes and type the link: `references`, + `supports` or `contradicts`. Similarity can tell that two notes are about the same thing; + only you know whether one backs the other up or argues against it. - **Searches honestly.** Keyword and semantic search together. When the vault holds no evidence for a query, B2 says "no matches" instead of showing its nearest guesses. -- **Answers from your notes, with sources.** Ask a question and get a streamed answer grounded - only in your notes, with each `[n]` pointing back to the passage it came from. Nothing about - a chat is stored. +- **Answers from your notes, with sources.** Ask a question and get a streamed answer + grounded only in your notes, each `[n]` pointing back to the passage it came from. - **Works with agents.** Every CLI command has a `--json` form, and the vault is plain Markdown on disk, so an agent can search, read and walk the graph the way you do. - **Is a real editor.** Live-preview Markdown, `[[wikilink]]` completion, images, a graph view, a file tree over your real folders, and full keyboard control with rebindable keys. - Indexing is automatic: the app watches the folder and keeps up with edits made anywhere. ## What it promises -- **Your notes stay yours.** The vault is a folder of Markdown that reads fine in Obsidian or - any editor. B2's index lives in `.b2/` and can be deleted and rebuilt at any time. -- **No surprise writes.** Opening, reading and indexing a vault change nothing. B2 writes only - when you ask it to (a link, a move, a save), and it never writes into the body of a note on - its own. -- **Local first.** Embeddings run on your machine. Chat uses a local model through Ollama by - default; a cloud model is something you choose. -- **Refactor without fear.** Move or rename a note through B2 and every `[[wikilink]]` to it is +- **Your notes stay yours.** The vault is a folder of Markdown that reads fine in Obsidian + or any editor. B2's index lives in `.b2/` and can be deleted and rebuilt at any time. +- **No surprise writes.** Opening, reading and indexing a vault change nothing. B2 writes + only when you ask it to (a link, a move, a save), and never into the body of a note on its + own. +- **Local first.** Embeddings run on your machine. Chat uses a local model through Ollama + by default; a cloud model is something you choose. +- **Refactor without fear.** Move or rename a note through B2 and every link to it is rewritten to match. -## Status - -The build-out is done: the engine, the `b2` CLI, the desktop app and grounded chat all work end -to end. The next phase is using B2 every day and letting that use decide what comes next. New -work starts as an [`observed` issue](https://github.com/AlteredCraft/B2/issues?q=label%3Aobserved): -what I was doing, what I expected, what happened. B2 runs from source on macOS today (the -one tested platform); an installable build is [#23](https://github.com/AlteredCraft/B2/issues/23). - ## Get started ```bash make doctor # checks Rust, Node, the Tauri CLI and the build toolchain -make init # one-time: download the embedding model +make init # one time: download the embedding model B2_VAULT_PATH=~/notes make app # open the desktop app on a folder of Markdown ``` -Point it at an existing Obsidian vault or any folder of `.md` files. Without `B2_VAULT_PATH`, -the app asks you to pick a folder and remembers it. For chat, install +Point it at an existing Obsidian vault or any folder of `.md` files. For chat, install [Ollama](https://ollama.com) and run `ollama pull gemma4:e2b`. The same engine runs from the terminal: @@ -71,36 +60,35 @@ b2 -C ~/notes similar notes/memory b2 -C ~/notes ask "what have I written about sleep and memory?" ``` -The **[Quick start](docs/quickstart.md)** walks through the rest in about ten minutes, and has -every command, setting and environment variable. +The [quick start](docs/quickstart.md) has the walkthrough, every command and every setting. +B2 runs from source on macOS today; an installable build is +[#23](https://github.com/AlteredCraft/B2/issues/23). ## How it's built A Rust workspace. `b2-core` is the whole engine: it turns a folder of Markdown into a SQLite index (full-text search, vectors and a typed link graph) that is a pure projection of the -files. The `b2` CLI and the Tauri desktop app are two thin front ends over the same typed API. -The real models sit behind small seams (`b2-embed` for embeddings, `b2-llm` for chat), so a -better model drops in without a redesign, and the engine is tested without them. +files. The `b2` CLI and the Tauri desktop app are two thin front ends over the same typed +API. The real models sit behind small seams (`b2-embed` for embeddings, `b2-llm` for chat) +and the engine is tested without them. Two ideas shape every decision: -- **The vault is the truth; the index is disposable.** `index = projection of (the vault - directory)`. Drop the index, rebuild it, get the same thing back. -- **Build for tomorrow's model.** Do the least today's model needs, behind a seam, so the next - one is an upgrade rather than a rewrite. +- **The vault is the truth; the index is disposable.** Drop the index, rebuild it, get the + same thing back. +- **Build for tomorrow's model.** Do the least today's model needs, behind a seam, so the + next one is an upgrade and not a rewrite. ## Docs | Doc | What it covers | |---|---| -| [Quick start](docs/quickstart.md) | Set up and use B2: the walkthrough, commands, config, environment variables. | -| [Search and similarity](docs/search-and-similarity.md) | What search and the related-notes list do, in plain language. | -| [Architecture](docs/architecture.md) | The crates, the flows, the seams, and how the tests hold it up. | +| [Quick start](docs/quickstart.md) | Using B2: the walkthrough, commands, config, environment variables. | +| [Data model](docs/data-model.md) | What a vault holds: notes, links, the verbs, resources. | +| [Architecture](docs/architecture.md) | How it's built: the crates, the index, the flows. | | [Invariants](docs/invariants.md) | What must always be true, cited by id. Wins any conflict. | -| [Data model](docs/data-model.md) | What a note and a connection are, in plain Markdown. | -| [Index engine](docs/index-engine.md) | How the index is built and queried. | -| [Evals](docs/evals.md) | How search, discovery and chat quality are measured, outside CI. | -| [ADRs](ADRs/README.md) | Why each key decision reads the way it does. | +| [Decisions](docs/decisions.md) | The few key decisions, and why. | +| [Evals](docs/evals.md) | How search, discovery and chat quality are measured. | ## Develop @@ -110,6 +98,8 @@ make ci # exactly what GitHub Actions runs; run it before pushing make # every target, grouped ``` -There is no git hook; CI is the enforcement. Contributor and agent ground rules are in -[CLAUDE.md](CLAUDE.md), and the desktop crate has its own in +The build-out is done, so new work starts from daily use: an +[`observed` issue](https://github.com/AlteredCraft/B2/issues?q=label%3Aobserved) saying what +I was doing, what I expected and what happened. Ground rules for contributors and agents +are in [CLAUDE.md](CLAUDE.md), with the desktop crate's own in [crates/b2-desktop/CLAUDE.md](crates/b2-desktop/CLAUDE.md). diff --git a/crates/b2-desktop/CLAUDE.md b/crates/b2-desktop/CLAUDE.md index 57ae838..ee120b9 100644 --- a/crates/b2-desktop/CLAUDE.md +++ b/crates/b2-desktop/CLAUDE.md @@ -1,133 +1,126 @@ # CLAUDE.md: `b2-desktop` and `ui/` -Guidance for agents and humans working on the desktop app: this crate (the Tauri host) and -[`ui/`](../../ui) (the frontend). It adds to the [root CLAUDE.md](../../CLAUDE.md); everything -there still applies. +Rules for the desktop app: this crate (the Tauri host) and [`ui/`](../../ui) (the frontend). +They add to the [root CLAUDE.md](../../CLAUDE.md); everything there still applies. ## What the app is for -The desktop app is where B2 gets used every day. It is the notes app that replaces Obsidian, so -it has to be a good place to read and write first, and a research tool second: the related notes, -**Why?**, typed links and cited chat sit *beside* the note, never in the way of it. A citation -opens its evidence in the centre pane, next to the answer. +The desktop app is where B2 gets used every day. It replaces Obsidian, so it has to be a +good place to read and write first and a research tool second: related notes, **Why?**, +typed links and cited chat sit *beside* the note, never in the way of it. -Most `observed` issues will land here, because this is where friction is felt. Prefer making a +Most `observed` issues land here, because this is where friction is felt. Prefer making a daily path smoother over adding a new surface. ## The one rule: hold no engine logic -This crate is the second **dumb adapter** over the [`Vault`](../b2-core/src/vault.rs) façade, -the GUI sibling of [`b2-cli`](../b2-cli). Every `#[tauri::command]` is: deserialize the arguments, -call one `Vault` method, serialize the result. +This crate is the second dumb adapter over the [`Vault`](../b2-core/src/vault.rs) façade +(E3). Every `#[tauri::command]` is: deserialize the arguments, call one `Vault` method, +serialize the result. - A branch, loop or rule that wants to live in a handler belongs in `b2-core`, behind a new façade method. Add the method when a command needs it; don't pre-build. - Return the CLI's existing `--json` view types. Don't define a parallel set of DTOs. - `b2-core` never learns about Tauri, webviews or the UI. +- Tauri IPC is the only transport. This crate runs no HTTP server. -Why it matters: two adapters over one contract can't drift, a fix in the core fixes both, and the -app inherits the engine's tests instead of needing its own copy (invariant E3). **Smell test:** a -command body longer than "parse, call, return", or a reach for a `b2-core` internal that isn't on -`Vault`, means a façade method is missing. +**Smell test:** a command body longer than "parse, call, return", or a reach for a `b2-core` +internal that isn't on `Vault`, means a façade method is missing. -The same holds in `ui/`: the frontend decides how things look and how keys move, never what the -engine means. Pure logic lives in small tested modules (`treenav.ts`, `sidenav.ts`, `chat.ts`, -`droplink.ts`, `keymap.ts`, …) that run under node; `main.ts` is the wiring. Each module's header -comment explains its reasoning; read it before changing the module. +The same holds in `ui/`. The frontend decides how things look and how keys move, never what +the engine means. Pure logic lives in small tested modules (`treenav.ts`, `sidenav.ts`, +`chat.ts`, `droplink.ts`, `keymap.ts`) that run under node, and `main.ts` is the wiring. +Read a module's header comment before changing it. ## Wiring -- **Embedder, as in the CLI.** Reads open with the fake embedder; anything that embeds a query or - writes vectors opens the real model and fails fast with "run `b2 init`". The save, new-note, - frontmatter-save and project paths are deliberately model-free, so editing never waits on a - model; the background embed fills vectors afterwards. -- **Errors reach the webview generic and actionable.** Map façade errors as the CLI's - `user_message` does, through a `thiserror` enum. The full internal detail is always logged - host-side (`log_internal`, from `CmdError`'s `Serialize`), so a failure is diagnosable without - `B2_DEBUG`. -- **Chat provider, the embedder's sibling** (`src/chat.rs`). Resolution is - `b2_llm::LlmConfig::from_env` with Settings layered on top, the way a CLI flag would be. Chat - config lives beside the remembered vault, never in it. -- **The API key lives in the macOS Keychain** (`src/keychain.rs`), never in a file. `B2_LLM_API_KEY` - overrides it; a Keychain that refuses must not break chat (the key stays in force for the - session); and the key never crosses back to the webview. -- **Logging** is installed in `main` (`logging::init_logging`), with a non-blocking writer because - this process is long-lived. `main` holds the returned guard for the whole run. - -## Keyboard: fully operable without a mouse (invariant K1) - -**No action is reachable only by pointer.** A click-only, right-click-only or drag-only gesture is -a bug. Every new surface owes four things, cheap now and expensive to retrofit: - -1. **Reachable:** a real `