docs: document the repository graph and project memory

docs/agents.md

@@ -98,6 +98,27 @@ physically cannot call `approve`, and a supervisor physically cannot call
 `submit`. This is what makes the loop safe to drive from "untrusted"
 agents.
 
+### Retrieval tools
+
+When the optional [repository graph](/docs/repository-graph) is configured,
+**every** role — supervisor, executor, and the unfiltered server — additionally
+gets the same six **read-only** retrieval tools:
+
+| Tool | Domain |
+|---|---|
+| `repository_graph_status` | Repository graph availability, freshness, task-view status |
+| `repository_search` | Bounded ranked structural search |
+| `repository_context` | Bounded context assembly, with opt-in verified snippets |
+| `project_memory_status` | [Project memory](/docs/project-memory) revision, freshness, source policy |
+| `project_context_search` | Bounded search across `repository`, `memory`, or `all` |
+| `project_context` | Bounded federated context assembly |
+
+None of them resolves, claims, renews, or mutates a task lease, and none of
+them builds an index. Their output is never injected into task or review
+prompts — an agent has to ask. An executor working in a managed worktree is
+answered from that task's own pinned baseline-plus-overlay view, never from
+canonical `latest`.
+
 ## Skill files
 
 `ferrus init` creates skill files under your `--agents-path`

docs/configuration.md

@@ -35,6 +35,9 @@ heartbeat_interval_secs = 30    # how often agents should call heartbeat
 [spec]
 directory = "docs/specs"        # where /create_spec writes approved specs
 
+[repository_graph]              # optional; NOT scaffolded by `ferrus init`
+enabled = true                  # opt into the local repository graph + project memory
+
 [hq.supervisor]
 agent = "claude-code"    # agent for supervisor/reviewer role: claude-code | codex | qwen-code | goose | opencode
 model = ""                # optional override; empty = agent default
@@ -108,6 +111,114 @@ in `ferrus.db`.
 directory = "docs/specs"  # any path inside the project; created on first write
 ```
 
+## `[repository_graph]`
+
+The optional [repository graph](/docs/repository-graph) and
+[project memory](/docs/project-memory) indexes. This namespace is **not written
+by `ferrus init`** — add it yourself to opt in:
+
+```toml
+[repository_graph]
+enabled = true
+```
+
+Everything else has a default. Omitting the namespace entirely means the
+optional indexes stay off and ordinary orchestration is unaffected.
+
+### Namespaces
+
+| Namespace | Purpose | Affects snapshot identity |
+|---|---|---|
+| `[repository_graph]` | `enabled`, backend selection | No — operational |
+| `[repository_graph.source]` | Included content, untracked/generated/vendor policy, ordered ignore rules, sensitive paths | **Yes** |
+| `[repository_graph.analyzers]` | Enabled extractors and their semantic settings | **Yes** |
+| `[repository_graph.index_limits]` | Limits that can include, skip, or truncate a file during extraction | **Yes** |
+| `[repository_graph.query_limits]` | Result, byte, depth, duration, snippet, and diagnostic budgets | No |
+| `[repository_graph.retention]` | Snapshot and failed-build cleanup policy | No |
+| `[repository_graph.memory]` | Authorized project-memory sources and extraction policy | Memory revision only |
+| `[repository_graph.semantic]` | Reserved for future embedding/chunking policy | Semantic projection only |
+| `[repository_graph.remote]` | Reserved endpoint, credential reference, upload policy | No |
+| `[repository_graph.telemetry]` | Privacy-safe query metrics | No |
+
+The "affects snapshot identity" column is the one that matters day to day:
+changing a query budget or retention rule **never** invalidates your index,
+while changing what gets indexed does. Configuration is hashed as a canonical
+semantic projection, not as raw TOML — whitespace, key ordering, and explicitly
+writing out a default all produce the same digest. Credentials, tokens, and
+endpoints never participate in identity and never appear in diagnostics.
+
+### `[repository_graph.source]`
+
+```toml
+[repository_graph.source]
+include = ["**/*"]                         # set-like; order is not significant
+rules = [".git/**", ".ferrus/**"]          # ordered ignore/negation rules; order IS significant
+sensitive = [                              # never read, never extracted, never uploaded
+    "**/.env", "**/.env.*",
+    "**/*.key", "**/*.pem", "**/*.p12", "**/*.pfx",
+    "**/id_rsa", "**/id_ed25519",
+]
+include_untracked = true
+include_generated = false
+include_vendor = false
+```
+
+Sensitive-path policy is applied **before** extraction, not after.
+
+### `[repository_graph.index_limits]`
+
+Bounds one indexing pass. Defaults:
+
+| Key | Default |
+|---|---|
+| `max_files` | `100000` |
+| `max_directories` | `100000` |
+| `max_file_bytes` | `2097152` (2 MiB) |
+| `max_total_bytes` | `536870912` (512 MiB) |
+| `max_facts_per_file` | `100000` |
+| `max_parser_duration_ms` | `2000` |
+| `max_resolved_relationships` | `1000000` |
+| `max_resolver_duration_ms` | `10000` |
+| `max_diagnostics` | `1000` |
+
+### `[repository_graph.query_limits]`
+
+Bounds every CLI and MCP read. The configured cap always wins over a caller's
+requested budget; exceeding one returns the deterministic prefix that fits plus
+an explicit truncation reason.
+
+| Key | Default |
+|---|---|
+| `max_results` | `100` |
+| `max_bytes` | `262144` (256 KiB) |
+| `max_snippet_bytes` | `32768` (32 KiB) |
+| `max_depth` | `3` |
+| `max_duration_ms` | `2000` |
+| `max_diagnostics` | `50` |
+
+### `[repository_graph.retention]`
+
+```toml
+[repository_graph.retention]
+max_snapshots = 5
+max_failed_builds = 10
+```
+
+Snapshots pinned by an active task are retained regardless.
+
+### `[repository_graph.telemetry]`
+
+```toml
+[repository_graph.telemetry]
+enabled = true
+```
+
+Emits one structured tracing metric per query: tool name, repository/task/run/
+snapshot identities, freshness, duration, result count, response bytes,
+truncation reason, diagnostic count, and error category. The metric types
+**cannot represent** request text, filters, repository paths, snippets, or
+source bodies. Off by default.
+
 ## `[hq.supervisor]` and `[hq.executor]`
 
 Which coding agent plays which role. Change these to swap backends without
@@ -161,6 +272,8 @@ intent and run artifacts, not a mirrored state machine.
 |---|---|
 | `project.toml` | Project id, name, workspace path, `.ferrus` path, git metadata, timestamps, schema version |
 | `ferrus.db` | SQLite source of truth for tasks, runs, events, leases, counters, and project runtime state |
+| `repo-graph.db` | Derived [repository graph](/docs/repository-graph) facts. Optional and rebuildable — safe to delete |
+| `project-memory.db` | Derived [project memory](/docs/project-memory) revisions. Optional and rebuildable — safe to delete |
 | `archive/specs/<spec-slug>-<closed-at>/` | Completed-spec archives written by `/archive-spec`: `manifest.toml`, a copy of `spec.md`, and the relocated `tasks/` and `runs/` artifacts |
 | `logs/` | Reserved for machine-local logs that should not be committed |
 

docs/project-memory.md

@@ -0,0 +1,203 @@
+---
+id: project-memory
+title: Project memory
+sidebar_position: 7
+slug: /project-memory
+---
+
+# Project memory
+
+The [repository graph](/docs/repository-graph) knows what your code looks like
+**now**. Project memory knows what your project **decided** — approved
+outcomes, milestones, deviations, validation evidence, follow-up work — so a
+fresh agent doesn't re-litigate a question that was settled three specs ago.
+
+It is a second derived, rebuildable index in its own SQLite sidecar
+(`project-memory.db`). It is not orchestration state: deleting it removes no
+tasks, runs, archives, specifications, or graph snapshots. Like the graph, it
+works entirely offline — no network call, no LLM, no embedding model.
+
+## Enable and build
+
+Project memory shares the repository graph's configuration namespace, so enable
+the graph first:
+
+```toml
+[repository_graph]
+enabled = true
+```
+
+Then:
+
+```bash
+ferrus graph memory index          # incremental
+ferrus graph memory index --full   # bypass fragment reuse; also replaces incompatible storage
+ferrus graph memory status
+ferrus graph memory status --json
+```
+
+```text
+Project memory: NotBuilt
+Revision: none
+Freshness: Unknown
+Next: ferrus graph memory index
+```
+
+Indexing is incremental — unchanged authorized sources reuse cached fragments,
+and a no-change run does not advance the published generation. Publication is
+atomic: a failed or interrupted build leaves the previous revision readable.
+
+## What gets indexed — and what never does
+
+This is the part worth reading twice. Memory ingestion runs on an **explicit
+allowlist**, and the presence of a file under `.ferrus/` or in an archive is
+*not* permission to read it.
+
+Enabled by default:
+
+| Source category | Content boundary | Sensitivity |
+|---|---|---|
+| Specification structure | Tracked headings and milestone metadata | Curated |
+| Approved outcome | The approved `## Outcome` section | Curated |
+| Archive manifest | Archive identity and counts | Operational metadata |
+| Runtime provenance | Terminal task, run, milestone, status, and check **identities** | Operational metadata |
+
+Disabled and treated as sensitive — raw task descriptions, submissions,
+reviews, patches, check logs, questions, answers, consultations, and
+integration-error bodies. Runtime provenance opens `ferrus.db` read-only and
+keeps only IDs and statuses; raw payloads, task paths, agent names, PIDs,
+workspace paths, and failure reasons do not cross the adapter.
+
+Use JSON status to audit every category and its effective policy:
+
+```bash
+ferrus graph memory status --json
+```
+
+:::warning[Curated does not mean public]
+Approved `## Outcome` sections are the one thing ferrus copies verbatim into
+memory. Keep them concise and free of secrets.
+:::
+
+## Entities and relationships
+
+Memory covers specifications, milestones, outcomes, decisions, deviations,
+validation evidence, follow-up work, and task/run references, related through
+typed edges: `contains`, `implements`, `validates`, `supersedes`, `concerns`,
+`touches`, `follows_up`. These are history semantics and are deliberately
+independent of repository dependency edges.
+
+Every entity and relationship carries project scope, memory revision, source
+category, a portable locator, source fingerprint, extractor identity and
+version, evidence span or record ID, resolution state, confidence, and
+timestamps.
+
+## Querying
+
+Repository-only is the default — nothing changes unless you ask for it:
+
+```bash
+ferrus graph search RuntimeTaskContext                     # repository only
+ferrus graph search "bounded retrieval" --domain memory    # memory only
+ferrus graph context --milestone rg4.6 --domain memory
+ferrus graph context --milestone rg4.6 --domain all        # federated
+```
+
+Memory context accepts several seed types:
+
+```bash
+ferrus graph context --domain memory --memory-entity <id>
+ferrus graph context --domain memory --milestone rg4.6
+ferrus graph context --domain memory --task <task-id>
+ferrus graph context --domain memory --run <run-id>
+```
+
+Federated responses report both identities independently:
+
+```text
+Repository: snapshot=snapshot:7b39a635... freshness=Stale
+Memory: revision=revision:... freshness=Fresh
+```
+
+### What `--domain all` does *not* do
+
+It does not merge the two stores, and it does not turn history into a claim
+about current code. It resolves the repository snapshot and memory revision
+separately, loads only the exact link set for those two revisions, and crosses
+domains **only** through evidence-backed links. Current repository evidence
+wins equal-rank tie-breaking, so historical context supplements source rather
+than displacing it. `Fresh` in one domain says nothing about the other.
+
+## Repository links: resolved, stale, unresolved
+
+A memory fact can point at code, but only on explicit evidence — the tracked
+specification path, the repository-relative spec path in an approved archive
+manifest, curated inline references written as `` `path:src/lib.rs` `` or
+`` `symbol:rust:function:src/lib.rs:run` ``, or baseline/materialized snapshot
+IDs attached to terminal task and run metadata. Changed-path evidence is
+computed from file identities in two authorized snapshots — raw patches and
+artifact bodies are never read.
+
+| State | Meaning |
+|---|---|
+| `resolved` | Exact path or unique semantic-key match in the selected snapshot |
+| `stale` | Resolved in an earlier revision, no longer matching the selected snapshot; retains the historical target |
+| `unresolved` | Never matched, origin unavailable, or ambiguous semantic key |
+
+Stale and unresolved links stay labelled and are presented as historical
+evidence only. Links are **never** promoted through text similarity or LLM
+inference, and link sets are stored separately from semantic memory revisions —
+so reindexing unchanged memory against a new repository snapshot does not
+rewrite memory facts or change the revision ID.
+
+## Freshness and the archive lifecycle
+
+`/archive-spec` is the only workflow that writes approved `## Outcome` content.
+After the archive transaction commits, ferrus attempts an incremental memory
+refresh — deliberately **outside** the archive's critical path, so a refresh
+failure can never undo or fail a successful archive.
+
+Repository and memory freshness are independent. When an authorized source
+changes, queries keep the last published revision readable, report memory as
+stale, and recommend:
+
+```bash
+ferrus graph memory index
+```
+
+CLI status compares the current authorized source manifest exactly.
+Latency-bounded MCP retrieval may report freshness as `unknown` when it cannot
+perform that comparison safely — `unknown` is not a claim that the revision is
+fresh.
+
+## What agents get
+
+Supervisor and Executor see the same read-only surface:
+
+| Tool | Purpose |
+|---|---|
+| `project_memory_status` | Availability, revision, freshness, source policy, counts |
+| `project_context_search` | Bounded ranked search across the selected domain |
+| `project_context` | Bounded evidence-backed context assembly |
+
+Every MCP request must state its domain — `repository`, `memory`, or `all` —
+explicitly. The tagged shape makes it impossible to silently broaden a
+repository-only request. None of these tools builds an index, authors an
+`## Outcome`, or touches task lifecycle state.
+
+## Operational notes
+
+- Treat stale or unresolved repository links as historical evidence, never as a
+  current-code relationship.
+- Do **not** use `project-memory.db` as a backup. Rebuild it from tracked specs
+  and registered runtime metadata.
+- Status reports retention diagnostics (total and historical revisions,
+  terminal builds not backing the current publication, link-set counts). They
+  are diagnostics only — ferrus does not silently delete history. If the
+  sidecar grows unexpectedly, `--full` is the explicit cleanup path.
+
+## See also
+
+- [Repository graph](/docs/repository-graph) — structural indexing and retrieval
+- [Specs & Milestones](/docs/spec-and-milestones#closing-out-a-spec-archive-spec) — where `## Outcome` comes from
+- [Configuration](/docs/configuration#repository_graph)

docs/quickstart.md

@@ -102,6 +102,30 @@ summary and files the spent task/run artifacts away — see the
 Press **Ctrl+C** twice within 2 seconds to exit HQ.
 :::
 
+## 5. (Optional) Index your repository
+
+ferrus can build a local structural index of your codebase so agents navigate
+it by symbol and relationship instead of re-grepping it on every spawn. It is
+opt-in — add the namespace to `ferrus.toml`:
+
+```toml
+[repository_graph]
+enabled = true
+```
+
+Then build it:
+
+```bash
+ferrus graph index
+ferrus graph status
+ferrus graph search MyType --limit 10
+```
+
+The index lives in its own machine-local SQLite sidecar, is rebuildable, and is
+safe to delete at any time — an absent index never blocks the task loop. See
+[Repository graph](/docs/repository-graph) and
+[Project memory](/docs/project-memory).
+
 ## What happens next
 
 ```text