Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions SECURITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ The plugin can optionally start its own HTTP server (`externalApiEnabled`, defau

- **Loopback by default**: the server binds `127.0.0.1:8790` (`externalApiHost` / `externalApiPort`, or persisted panel settings). Rebinding to a non-loopback address exposes the store to your network and is an explicit operator decision.
- **Token authentication**: every route except `GET /health` requires `Authorization: Bearer <token>`; the token is auto-generated on first use, persisted in the plugin settings store and verified with `timingSafeEqual`. To rotate it, clear the persisted value (settings kv key `external_api`) and let the next boot mint a new one — there is no rotate endpoint yet.
- **Surface**: memories read/write (`GET/POST /memories`, `GET/PUT/DELETE /memories/:id`, `GET /search`, `GET /context`, `GET /status`), plus profile/rules **reads** (`GET /profile`, `GET /rules`), `POST /bootstrap` and `POST /maintenance/reclaim` (the host-side twin `/api/dsh-mneme/maintenance/reclaim` also answers `GET` for dry-run previews). Writes go through the same dedupe + quality-filter path as in-conversation saves. The full route list lives in [docs/DAEMON.md](dsh-mneme/docs/DAEMON.md).
- **Surface**: memories read/write (`GET/POST /memories`, `GET/PUT/DELETE /memories/:id`, `GET /search`, `GET /context`, `GET /status`), profile/rules **reads** (`GET /profile`, `GET /rules`), `GET /memory/provenance`, `POST /bootstrap` and `POST /maintenance/reclaim` (the host-side twin `/api/dsh-mneme/maintenance/reclaim` also answers `GET` for dry-run previews). Writes go through the same dedupe + quality-filter path as in-conversation saves. The full route list lives in [docs/DAEMON.md](dsh-mneme/docs/DAEMON.md).
- **CLI**: `bin/cli.mjs` is a thin client over this API and stores its credentials in `~/.dsh-mneme/cli.json` - treat that file like any credential file.

---
Expand Down Expand Up @@ -154,7 +154,7 @@ The token is a lock on the **write** surface, not a wall around the whole API.
- **The real boundary is loopback + the DSH host listener.** The host-side routes (`/api/dsh-mneme/*`) are mounted on the DSH port, which binds `127.0.0.1` and is fenced by the host against DNS rebinding and cross-site browser requests. That fence does **not** establish identity: any local process, and any page running in the host's own origin, is inside the perimeter.
- **Authentication is opt-in and fail-open.** `apiToken` defaults to empty; while it is empty, `isAuthorized()` returns true for every request — reads *and* writes. Setting a token gates the write routes and the sensitive reads; it does not gate plain reads.
- **Open reads (no token even when one is configured)**: `GET /`, `/info`, `/version-check`, `/list`, `/recall-stats`, `/search`, `/semantic`, `/semantic/llm-audit`, `/semantic/llm-audit/stats`, `/semantic/graph/*`, `/entities`, `/memories/entities`, `/dream-status`, `/inject-status`, `/inject-preview`, `/pipeline`, `/candidates`, `/llm-providers`, plus the `GET`/`HEAD` branches of `/profile`, `/rules`, `/persona`, `/mode`, `/features`, `/commands`, `/nudge`. Treat them as readable by anything local.
- **Gated routes (when a token is configured)**: every write — `/create`, `/update`, `/delete`, `/import`, `/vector-config`, `/vector-reindex`, `/runtime/provision`, `/maintenance/reclaim`, `/test-model`, `/conflicts/resolve`, `/profile/extract` (POST-only: one host-LLM call that returns a profile **draft**; other methods get 405 before any side effect), and the non-`GET` branches of `/profile`, `/rules`, `/persona`, `/mode`, `/features`, `/commands`, `/nudge` — plus four reads: `/export`, `/external-api`, `/conflicts`, `/health`.
- **Gated routes (when a token is configured)**: every write — `/create`, `/update`, `/delete`, `/import`, `/vector-config`, `/vector-reindex`, `/runtime/provision`, `/maintenance/reclaim`, `/test-model`, `/conflicts/resolve`, `/profile/extract` (POST-only: one host-LLM call that returns a profile **draft**; other methods get 405 before any side effect), and the non-`GET` branches of `/profile`, `/rules`, `/persona`, `/mode`, `/features`, `/commands`, `/nudge` — plus five reads: `/export`, `/external-api`, `/conflicts`, `/health`, `/memory/provenance`. That last one is gated because it returns the **verbatim raw text** of distilled conversation windows (opt-in via `distillRecords.enabled`); chained with the open list route it would otherwise let any local process enumerate ids and then pull that text back, which reaches the same result as `/export` in a different shape.
- **The standalone data plane** (`127.0.0.1:8790`) is the opposite shape: every route except `GET /health` requires its own Bearer token. Pointing `externalApiHost` at a non-loopback address therefore turns it into a plaintext, token-only HTTP surface — an explicit operator decision.
- **Consequence**: if you expose the plugin beyond loopback (reverse proxy, non-loopback bind), set `apiToken` *and* check that the plain reads above are acceptable to expose — they are not covered by it.
- **How this list stays true**: every route declares its own `policy` (`open` / `token` / `token-write`) at registration, and `createRouteRegistrar` throws when a route has none or reuses a path. A new route that forgets its classification **fails to boot** instead of silently opening. The two lists above are compared against the routes actually mounted by `dsh-mneme/test/security-doc.test.js` in both directions, so adding a route without updating this section fails CI rather than quietly drifting — the matrix is derived from code, not from anyone's memory.
Expand Down Expand Up @@ -432,7 +432,7 @@ This project is licensed under the **MIT License**. See [LICENSE](https://github

- **默认只绑回环**:服务绑 `127.0.0.1:8790`(`externalApiHost` / `externalApiPort`,或面板里持久化的设置)。改绑非回环地址等于把记忆库暴露给你的网络,这是运营者的显式决定。
- **Token 鉴权**:除 `GET /health` 外每条路由都要 `Authorization: Bearer <token>`;token 首次使用时自动生成、持久化在插件设置库、用 `timingSafeEqual` 校验。要轮换就清掉持久化的值(设置 kv 键 `external_api`),下次启动会重新铸造——目前没有 rotate 端点。
- **路由面**:记忆读写(`GET/POST /memories`、`GET/PUT/DELETE /memories/:id`、`GET /search`、`GET /context`、`GET /status`),profile/rules 的**读**(`GET /profile`、`GET /rules`),以及 `POST /bootstrap` 与 `POST /maintenance/reclaim`(宿主侧孪生路由 `/api/dsh-mneme/maintenance/reclaim` 另外还答 `GET`,用于 dry-run 预览)。写入走与对话内保存相同的去重 + 质量过滤路径。完整路由表见 [docs/DAEMON.md](dsh-mneme/docs/DAEMON.md)。
- **路由面**:记忆读写(`GET/POST /memories`、`GET/PUT/DELETE /memories/:id`、`GET /search`、`GET /context`、`GET /status`),profile/rules 的**读**(`GET /profile`、`GET /rules`),`GET /memory/provenance`,以及 `POST /bootstrap` 与 `POST /maintenance/reclaim`(宿主侧孪生路由 `/api/dsh-mneme/maintenance/reclaim` 另外还答 `GET`,用于 dry-run 预览)。写入走与对话内保存相同的去重 + 质量过滤路径。完整路由表见 [docs/DAEMON.md](dsh-mneme/docs/DAEMON.md)。
- **CLI**:`bin/cli.mjs` 是这个 API 上的薄客户端,凭据存在 `~/.dsh-mneme/cli.json`——请把该文件当凭据文件对待。

---
Expand Down Expand Up @@ -535,7 +535,7 @@ Token 是**写**面上的锁,不是把整个 API 围起来的墙。
- **真正的边界是回环 + 宿主监听器。** 宿主侧路由(`/api/dsh-mneme/*`)挂在 DSH 端口上,该端口绑 `127.0.0.1`,并由宿主挡掉 DNS rebinding 与跨站浏览器请求。这道围栏**不建立身份**:任何本地进程、以及任何跑在宿主同源页面里的脚本,都在这道围栏之内。
- **鉴权是 opt-in 且 fail-open。** `apiToken` 默认为空;为空期间 `isAuthorized()` 对每个请求都返回 true——读和写都放行。配了 token 之后,它守的是写路由与敏感读,不守普通读。
- **开放读(即使配了 token 也不需要)**:`GET /`、`/info`、`/version-check`、`/list`、`/recall-stats`、`/search`、`/semantic`、`/semantic/llm-audit`、`/semantic/llm-audit/stats`、`/semantic/graph/*`、`/entities`、`/memories/entities`、`/dream-status`、`/inject-status`、`/inject-preview`、`/pipeline`、`/candidates`、`/llm-providers`,以及 `/profile`、`/rules`、`/persona`、`/mode`、`/features`、`/commands`、`/nudge` 的 `GET`/`HEAD` 分支。把它们当作「本地任何东西都能读」。
- **受门控的路由(配了 token 之后)**:全部写操作——`/create`、`/update`、`/delete`、`/import`、`/vector-config`、`/vector-reindex`、`/runtime/provision`、`/maintenance/reclaim`、`/test-model`、`/conflicts/resolve`、`/profile/extract`(只收 POST:花一次宿主 LLM 产一段画像**草稿**,其它方法在产生任何副作用之前就 405),以及 `/profile`、`/rules`、`/persona`、`/mode`、`/features`、`/commands`、`/nudge` 的非 `GET` 分支——外加四条读:`/export`、`/external-api`、`/conflicts`、`/health`。
- **受门控的路由(配了 token 之后)**:全部写操作——`/create`、`/update`、`/delete`、`/import`、`/vector-config`、`/vector-reindex`、`/runtime/provision`、`/maintenance/reclaim`、`/test-model`、`/conflicts/resolve`、`/profile/extract`(只收 POST:花一次宿主 LLM 产一段画像**草稿**,其它方法在产生任何副作用之前就 405),以及 `/profile`、`/rules`、`/persona`、`/mode`、`/features`、`/commands`、`/nudge` 的非 `GET` 分支——外加五条读:`/export`、`/external-api`、`/conflicts`、`/health`、`/memory/provenance`。最后这条受门控是因为它回的是蒸馏窗口的**逐字会话原文**(`distillRecords.enabled` 打开才有):否则把它和开放的那个列表路由串起来,任何本地进程都能先枚举 id、再按 id 把那些原文捞回去——可达结果与 `/export`(一次拿全库)同量级,只是换了形状。
- **独立数据面**(`127.0.0.1:8790`)是反过来的形状:除 `GET /health` 外每条路由都要自己的 Bearer token。因此把 `externalApiHost` 指向非回环地址,就等于把它变成一个「明文、只靠 token」的 HTTP 面——这是运营者的显式决定。
- **推论**:如果你把插件暴露到回环之外(反代、绑非回环地址),请设 `apiToken`,**并**核对上面那串开放读是否也能被看到——它们不受 token 保护。
- **这张表怎么保持为真**:每条路由在注册时自己声明 `policy`(`open` / `token` / `token-write`),`createRouteRegistrar` 在路由没声明或复用同一路径时直接抛错。新路由忘了归类会**启动失败**,而不是静默开门。上面两张清单还会被 `dsh-mneme/test/security-doc.test.js` 拿真实装配出来的路由做双向比对——加了路由却不更新这一节,CI 会红,而不是悄悄漂走。这张矩阵来自代码,不来自谁的记忆。
Expand Down Expand Up @@ -760,5 +760,5 @@ dsh-mneme:

---

*Last updated: 2026-10-18*
*Last updated: 2026-10-23*
*Policy version: 3.0*
20 changes: 20 additions & 0 deletions dsh-mneme/lib/api-standalone.js
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ import { TYPES } from "./store.js";
import { bootstrapFromDirectory } from "./bootstrap.js";
import { PACKAGE_VERSION } from "./version-check.js";
import { parseReclaimJson, readReclaimFields, reclaimReply, RECLAIM_UNAVAILABLE } from "./reclaim-http.js";
import { parseProvenanceQuery, provenanceReply } from "./provenance-http.js";

const DEFAULT_PORT = 8790;
const DEFAULT_HOST = "127.0.0.1";
Expand Down Expand Up @@ -414,6 +415,25 @@ export function createStandaloneApi({ service, store, config = {}, logger, setti
return;
}

// --- GET /memory/provenance: 记忆级溯源读出口(#399 写、#421 读)--------
// 与宿主内 /api/dsh-mneme/memory/provenance 同一份实现口径,就在
// src/provenance-http.js 里写一份(#411 的教训:两张皮整段复制必然漂移)。
// 一个链接一个窗口(区间 + 区间内原文行,按 seq 升序);没有链接的记忆返回
// 空 windows——「不是蒸馏来的」是有效答案,不是错误。
//
// 这张面本来就要 Bearer token(除 /health 外无例外),所以不涉及宿主侧那次
// open → token 的归类调整;daemon 与宿主共用 service,口径天然一致。
if (req.method === "GET" && pathname === "/memory/provenance") {
const parsed = parseProvenanceQuery(url.searchParams.get("id"));
if (!parsed.ok) {
sendJson(res, parsed.status, { error: parsed.error });
return;
}
const { status, body } = provenanceReply({ service, id: parsed.id, warn: (m) => logger?.warn?.(m) });
sendJson(res, status, body);
return;
}

// --- GET/PUT/DELETE /memories/:id --------------------------------------
const idMatch = pathname.match(/^\/memories\/([^/]+)$/);
if (idMatch) {
Expand Down
38 changes: 38 additions & 0 deletions dsh-mneme/lib/api.js
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ import { computeHeat } from "./heat.js";
import { EFFORT_REJECT_RE, describeStreamFailure, resolveRoute, withEffortFallback } from "./dream.js";
import { getInjectionSnapshot } from "./inject.js";
import { renderUserSettingsBlock } from "./user-settings.js";
import { parseProvenanceQuery, provenanceReply } from "./provenance-http.js";
import { describeLocalRuntime, publicRuntimeStatus } from "./runtime/loader.js";
import { hostModulesDir, provisionRuntime } from "./runtime/provision.js";
import { classify, fetchLatestVersion, PACKAGE_VERSION } from "./version-check.js";
Expand Down Expand Up @@ -1293,6 +1294,43 @@ export function createApi(ctx, service, settings, commands, embedder, semantic =
}
});

// --- 记忆级溯源读出口(#399 写、#421 读)-----------------------------------
// #399 把蒸馏窗口的原文落进 distill_records、把 memory → (session, seq 区间)
// 的归因写进 memory_provenance,但只做了写:库里有答案、service.provenanceFor
// 能查,外部(面板 / standalone API / 脚本)拿不到,「这条记忆是从哪几句蒸出来
// 的」只能开 sqlite 手写 JOIN。本端点补最后一段接线。
//
// 一个链接一个窗口:区间 + 区间里的原文行(按 seq 升序)。没有链接的记忆
// (memory_save / 工具写入 / 冷启动)返回空 windows——「不是蒸馏来的」本身是
// 有效答案,不是错误,所以不 404。
//
// policy 是 **token**,不是 open(#509 维护者评审定):records 带的是蒸馏窗口的
// 逐条会话原文,而 /list 是 open 且逐条带 id——两者串起来等于「先枚举 id、再按 id
// 捞回原文」,可达结果与 /export(一次拿全库,本仓 2026-10 审计后列 gated)同量级,
// 只是换了形状。抽稀不该靠「调用方不知道有这条路」。
//
// 口径只在 src/provenance-http.js 写一份,两个入口都调它(#411 的教训:两张皮
// 整段复制必然漂移)。
register({
kind: "exact",
path: "/api/dsh-mneme/memory/provenance",
policy: "token",
handler(req, res) {
try {
const url = new URL(req.url, "http://localhost");
const parsed = parseProvenanceQuery(url.searchParams.get("id"));
if (!parsed.ok) {
sendJson(res, parsed.status, { error: parsed.error });
return;
}
const { status, body } = provenanceReply({ service, id: parsed.id, warn: (m) => ctx.logger?.warn?.(m) });
sendJson(res, status, body);
} catch {
sendJson(res, 500, { error: "internal" });
}
}
});

// --- ego graph: 1-2 hop neighborhood of one entity (graph panel P1) ---
// Read-only like list/search/semantic, so it stays open when apiToken is set.
// BFS from the root entity over entity_relations (both directions; the
Expand Down
68 changes: 68 additions & 0 deletions dsh-mneme/lib/provenance-http.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
// 记忆级溯源读出口的 HTTP 口径 —— 唯一实现。
//
// 为什么要单独一个模块:这张脸有两个入口——宿主内的面板路由(src/api.js 的
// /api/dsh-mneme/memory/provenance)与 Bearer 数据面(src/api-standalone.js 的
// /memory/provenance)。两者底下是同一份 service(provenanceFor + distillRecordsFor),
// 差别只在鉴权与路径。历史上(#509 第一版)它们是整段复制的,于是「改一边忘一边」
// 在结构上就可能发生——#411 已经为 reclaim / share 各踩过一次,本模块照那次的结论办。
//
// 口径(错误码、两种「空」的区分、窗口形状)现在只写在这里,两个入口都调它。
// 真要再写第二份,test/provenance-http.test.js 的静态锁会直接转红。
//
// 这里只做协议层的事:解析 id、组装窗口、区分「没接线」与「没有来源」。一行数据
// 都不改——读取在 src/store.js,透传在 src/service.js。

/**
* 「溯源能力没接线」的错误码。headless / 旧装配下 service 上没有 provenanceFor 或
* distillRecordsFor,那是装配坏了,不是「这条记忆没有来源」——两种空必须分开读数,
* 否则调用方看到一个空 windows 会以为拿到了答案。
*/
export const PROVENANCE_UNAVAILABLE = "provenance-unavailable";

/** 溯源读取失败的 warn 文案。两个入口共用,日志里不会出现两种拼法。 */
export function provenanceWarnMessage(error) {
return `[dsh-mneme] memory provenance read failed: ${String(error)}`;
}

/**
* 解析 ?id。空 / 缺失 → 400;其余原样交给 service(id 不存在时返回空 windows,
* 那是有效答案,不是错误——读端点不替调用方判断记忆是否存在)。
*
* @returns {{ok: true, id: string} | {ok: false, status: 400, error: string}}
*/
export function parseProvenanceQuery(raw) {
const id = typeof raw === "string" ? raw.trim() : "";
if (!id) return { ok: false, status: 400, error: "missing-id" };
return { ok: true, id };
}

/**
* 组装答复:一个链接一个窗口(区间 + 区间内原文行,按 seq 升序)。
*
* 顺序敏感:503 要赢过空结果——装配没接线时报「这条记忆没有来源」是误导,
* 与 reclaimReply 的 maintenance-unavailable 同构。
*
* @param {{service: object, id: string, warn?: Function}} args
* @returns {{status: number, body: object}}
*/
export function provenanceReply({ service, id, warn = null }) {
if (typeof service?.provenanceFor !== "function" || typeof service?.distillRecordsFor !== "function") {
return { status: 503, body: { error: PROVENANCE_UNAVAILABLE } };
}
try {
const links = service.provenanceFor(id);
const windows = links.map((link) => ({
session_id: link.session_id,
seq_start: link.seq_start,
seq_end: link.seq_end,
created_at: link.created_at,
// 到期裁剪后区间可能变空:链接刻意保留并指向空区间(clearDistillRecords 的
// 语义),面板据此显示「原文已过期」而非「没有来源」。
records: service.distillRecordsFor(link.session_id, link.seq_start, link.seq_end)
}));
return { status: 200, body: { memoryId: id, windows } };
} catch (error) {
warn?.(provenanceWarnMessage(error));
return { status: 500, body: { error: "internal" } };
}
}
3 changes: 3 additions & 0 deletions dsh-mneme/lib/service.js
Original file line number Diff line number Diff line change
Expand Up @@ -2599,6 +2599,9 @@ export function createService({ store, mirror, config, onWrite, logger, document
saveDistillRecords: (records) => store.saveDistillRecords(records),
saveMemoryProvenance: (link) => store.saveMemoryProvenance(link),
provenanceFor: (memoryId) => store.provenanceFor(memoryId),
// #421 读出口:按 (session, seq 区间) 取蒸馏原文。与 provenanceFor 成对——
// 一个给区间、一个给区间里的原文,合起来才是「这条记忆从哪几句蒸出来的」。
distillRecordsFor: (sessionId, seqStart, seqEnd) => store.distillRecordsFor(sessionId, seqStart, seqEnd),
enqueue,
setDreamHook(fn) { dreamHook = fn; },
setSleepHook(fn) { sleepHook = fn; },
Expand Down
Loading