Skip to content
81 changes: 81 additions & 0 deletions docs/operations/monitoring.md
Original file line number Diff line number Diff line change
Expand Up @@ -420,3 +420,84 @@ way. Bucket fill itself stays per host.
- [Telemetry](../features/observability/telemetry.md): the metric and span catalogue
- [Cost and usage](../features/observability/cost-and-usage.md) and
[audit](../features/observability/audit.md)

## Loom invocation observability

Loom observes actual HTTP operations and MCP tool calls through its existing
OpenTelemetry exporters. Each call observation has `loom.call.root=true`, a
unique `loom.call.attempt_id`, the operation, transport, outcome, and duration
in milliseconds. Provider and retrieval stages remain children of that call.

Transport and origin answer different questions. `loom.call.transport` is
`http` or `mcp`. `loom.call.origin` is the caller-reported `tool`, `hook`,
`worker`, or `unknown`. Absent or invalid origin stays unknown; MCP alone does
not prove that an agent selected the call. `caller_type` can separately name
`agent`, `plugin`, `sdk`, `cli`, or `worker`. Worker passes carry worker origin
but do not count as received requests, and the Langfuse worker filters still
apply. Origin and correlation metadata grant no permissions.

HTTP callers can supply these headers. Each value is independently optional;
identifiers accept at most 128 ASCII letters, digits, underscores, dots,
colons, or hyphens. Invalid identifiers are omitted.

| Header | Meaning |
| --------------------------------------------------------------------------------- | ------------------------------------------------------------------- |
| `X-Loom-Call-Origin` | Declared invocation origin |
| `X-Loom-Caller-Type` | Declared caller type |
| `X-Loom-Hook-Name`, `X-Loom-Hook-Event` | Known hook entrypoint and event |
| `X-Loom-Conversation-Id`, `X-Loom-Prompt-Id` | Optional supplied conversation and turn |
| `X-Loom-Tool-Call-Id` | Caller-supplied tool-call correlation; uniqueness is caller-defined |
| `X-Loom-Benchmark-Run-Id`, `X-Loom-Benchmark-Case-Id`, `X-Loom-Benchmark-Variant` | Evaluator correlation |
| `traceparent`, `tracestate` | Optional W3C parent context |

MCP callers supply the same snake-case fields in `params._meta.loom`, with
`origin` for the invocation origin; W3C context goes directly in `_meta`.
Per-call metadata takes precedence over request headers. For example:

```json
{
"name": "loom_op",
"arguments": { "op": "help" },
"_meta": {
"loom": {
"origin": "tool",
"caller_type": "agent",
"tool_call_id": "call-example",
"benchmark_run_id": "run-example",
"benchmark_case_id": "case-example",
"benchmark_variant": "with-memory"
}
}
}
```

The hook transport declares hook origin even without a prompt ID. Known hook
entrypoints also identify detached requests without borrowing another turn's
identity. Optional supplied IDs never change stored session scope, tenant
identity, authorization, or capture consent. Raw prompts, results and
credentials are not trace metadata.

Call observations cover HTTP operation refusals and MCP catalog, protocol,
and authorization-screening outcomes after the HTTP bearer gate. Invalid
JSON, unsupported HTTP protocol headers, whole-batch limits and bearer
refusals can precede MCP call observations. Private-work diagnostics are
suppressed by the existing export boundary. Trace sampling and span filters
still apply, so missing observations do not prove zero calls.

Count only observations marked `loom.call.root`, excluding HTTP envelopes,
batch entries and nested stages. Received retries have distinct attempt IDs.
A host tool-call ID can correlate several separate Loom requests from hook
fanout. Grouping attempts requires a caller guarantee that the supplied ID
identifies one logical Loom request within the caller's scope and remains
stable across its retries. Without that guarantee the logical count is unknown.
Structured benchmark metadata is filterable; it does not imply arbitrary
grouping or an answer-quality gain.

Sum call durations for accumulated call time. Concurrent requests overlap,
so this sum differs from elapsed wall time. A host prompt lifecycle is not
required for Loom invocation tracing. Valid supplied parent context can link
Loom calls to an external trace; absent context leaves an independent request.
Sum actual generation and embedding usage once on those observations, without
copying cost to the call parent. Missing usage or price stays unknown. Loom
provider usage is separate from external agent-host billing. Output length is
not measured model usage.
2 changes: 1 addition & 1 deletion integrations/claude-code-plugin/.claude-plugin/plugin.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "loom",
"displayName": "ClickHouse Loom",
"version": "1.2.45",
"version": "1.2.48",
"description": "Shared long-term memory and code intelligence for Claude Code, self-hosted on ClickHouse. MCP tools to save and recall decisions, preferences and corrections across sessions and teammates; lifecycle hooks that capture each session with no model call; a code index that answers where-is, who-calls and blast-radius questions with line-accurate units instead of whole-file reads.",
"author": {
"name": "ClickHouse"
Expand Down
15 changes: 15 additions & 0 deletions integrations/claude-code-plugin/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -511,3 +511,18 @@ Captures POST to the `observe.record` wire op and crystallization to
`observe.crystallize`. A failed POST is swallowed — the primer (`context.pack`)
and the safety model work regardless. See
[Claude Code](https://clickhouse.github.io/loom/features/integrations/claude-code/).

## Invocation attribution

Needs the next server release for invocation-correlation headers. Hook requests
declare hook origin and their entrypoint when known, including detached work.
The event can independently supply a conversation ID, `prompt_id` or `turn_id`,
tool-call ID and hook event. Missing IDs stay absent; conflicting prompt and
turn IDs are omitted. There is no shared latest-prompt binding. Trace headers
carry no prompt text, and identifiers pass through normal secret redaction.

This labels requests the plugin sends. The host's separate MCP client may
supply its own per-call metadata; absent origin stays unknown. Neither host
prompt lifecycle tracing nor external host billing is required. See
[Loom invocation observability](../../docs/operations/monitoring.md#loom-invocation-observability)
for the optional benchmark fields, sampling limits and accounting rules.
14 changes: 13 additions & 1 deletion integrations/claude-code-plugin/lib/loom-client.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ import { fileURLToPath } from "node:url";
// than that, and a shorter grace would kill a fire-and-forget post before it
// left a slow link.
import { DEFAULT_BOUND_MS, boundMs, rttCachePath } from "./hook-bounds.mjs";
import { entrypointHeaders, promptHeaders } from "./prompt-context.mjs";
import { readJsonFile, tmpStatePath, writeJsonFile } from "./tmp-state.mjs";

let _rttCache;
Expand Down Expand Up @@ -717,6 +718,9 @@ export function truncate(text, max = 2000) {
return s.slice(0, max - 14).trimEnd() + "\n…[truncated]";
}

const _hookEntryHeaders = entrypointHeaders(process.argv[1], fileURLToPath(new URL("../hooks/", import.meta.url)));
let _hookPromptHeaders = { ..._hookEntryHeaders };

// --- transport ----------------------------------------------------------------
// Every request the plugin makes goes through requestJson: one bound, one
// header set, one error shape. Nothing here throws or rejects.
Expand All @@ -736,6 +740,7 @@ export async function requestJson(
headers: {
...(body === undefined ? {} : { "content-type": "application/json" }),
...(token ? { authorization: `Bearer ${token}` } : {}),
..._hookPromptHeaders,
...headers,
},
...(body === undefined ? {} : { body: JSON.stringify(body) }),
Expand Down Expand Up @@ -1088,8 +1093,15 @@ export function readStdinJson() {
if (settled) return;
settled = true;
try {
resolve(normalizePayload(raw.trim() ? JSON.parse(raw) : {}));
const payload = normalizePayload(raw.trim() ? JSON.parse(raw) : {});
const candidate = { ..._hookEntryHeaders, ...promptHeaders(payload) };
// Identifiers still pass through redaction; no prompt or tool content is sent here.
_hookPromptHeaders = Object.fromEntries(
Object.entries(candidate).filter(([, value]) => redactSecrets(value) === value),
);
resolve(payload);
} catch {
_hookPromptHeaders = { ..._hookEntryHeaders };
resolve({});
}
};
Expand Down
24 changes: 24 additions & 0 deletions integrations/claude-code-plugin/lib/prompt-context.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
import { basename, dirname, resolve } from "node:path";

// Caller metadata describes hook requests even when the host supplies no prompt identity.
const ID = /^[a-zA-Z0-9_.:-]{1,128}$/;
const valid = (value) => typeof value === "string" && ID.test(value) && !/[\r\n]/.test(value);

export function promptHeaders(payload = {}) {
const headers = { "x-loom-call-origin": "hook", "x-loom-caller-type": "plugin" };
const { session_id: session, prompt_id: prompt, turn_id: turn } = payload;
if (valid(session)) headers["x-loom-conversation-id"] = session;
if (!(prompt && turn && prompt !== turn) && valid(prompt || turn)) {
headers["x-loom-prompt-id"] = prompt || turn;
}
if (valid(payload.tool_use_id)) headers["x-loom-tool-call-id"] = payload.tool_use_id;
if (valid(payload.hook_event_name)) headers["x-loom-hook-event"] = payload.hook_event_name;
return headers;
}

export function entrypointHeaders(entrypoint, hookDirectory) {
if (typeof entrypoint !== "string" || dirname(resolve(entrypoint)) !== resolve(hookDirectory)) return {};
const name = basename(entrypoint, ".mjs");
if (!/^loom-[a-z-]+$/.test(name)) return {};
return { "x-loom-call-origin": "hook", "x-loom-caller-type": "plugin", "x-loom-hook-name": name };
}
8 changes: 8 additions & 0 deletions integrations/claude-code-plugin/lib/usage.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,14 @@ export function extractTurns(objs, seen = new Set(), { cap = 200 } = {}) {
const u = msg.usage;
const id = String(msg.id || "");
if (!u || !id || next.has(id)) continue;
// Missing counters are unknown; a later transcript row may complete them.
if (
!Number.isSafeInteger(u.input_tokens) ||
u.input_tokens < 0 ||
!Number.isSafeInteger(u.output_tokens) ||
u.output_tokens < 0
)
continue;
const model = String(msg.model || "");
if (!model) continue;
const cc = u.cache_creation || {};
Expand Down
104 changes: 104 additions & 0 deletions integrations/claude-code-plugin/tests/prompt-context.test.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,104 @@
import "./temp-home.mjs";
import assert from "node:assert/strict";
import test from "node:test";
import { spawnSync } from "node:child_process";
import { promptHeaders } from "../lib/prompt-context.mjs";

test("host prompt IDs bind hook calls without capturing prompt text", () => {
const claude = promptHeaders({ session_id: "conversation-a", prompt_id: "prompt-a", prompt: "private text" });
const codex = promptHeaders({ session_id: "conversation-a", turn_id: "prompt-a" });
assert.equal(claude["x-loom-prompt-id"], "prompt-a");
assert.equal(codex["x-loom-prompt-id"], "prompt-a");
assert.equal(claude["x-loom-conversation-id"], "conversation-a");
assert.equal(claude["x-loom-call-origin"], "hook");
assert.equal(JSON.stringify(claude).includes("private text"), false);
});

test("optional IDs are independent and ambiguous prompt IDs are dropped", () => {
assert.equal(promptHeaders({ session_id: "conversation-a" })["x-loom-conversation-id"], "conversation-a");
assert.equal(promptHeaders({ prompt_id: "prompt-a" })["x-loom-prompt-id"], "prompt-a");
assert.equal(promptHeaders({ session_id: "c", prompt_id: "a", turn_id: "b" })["x-loom-prompt-id"], undefined);
assert.equal(promptHeaders({ session_id: "c", prompt_id: "bad\r\nheader" })["x-loom-prompt-id"], undefined);
assert.equal(promptHeaders({ session_id: "c", turn_id: "x".repeat(129) })["x-loom-prompt-id"], undefined);
});

test("identifiers reject trailing line breaks without losing hook origin", () => {
for (const suffix of ["\n", "\r", "\r\n"]) {
const badPrompt = promptHeaders({ session_id: "c", turn_id: `p${suffix}` });
const badSession = promptHeaders({ session_id: `c${suffix}`, turn_id: "p" });
assert.equal(badPrompt["x-loom-prompt-id"], undefined);
assert.equal(badSession["x-loom-conversation-id"], undefined);
assert.equal(badPrompt["x-loom-call-origin"], "hook");
const headers = promptHeaders({ session_id: "c", turn_id: "p", tool_use_id: `call${suffix}` });
assert.equal(headers["x-loom-tool-call-id"], undefined);
}
});

test("simultaneous turns and tool IDs stay separate", () => {
const a = promptHeaders({ session_id: "c", turn_id: "a", tool_use_id: "call-a", hook_event_name: "PreToolUse" });
const b = promptHeaders({ session_id: "c", turn_id: "b", tool_use_id: "call-b", hook_event_name: "PostToolUse" });
assert.equal(a["x-loom-prompt-id"], "a");
assert.equal(b["x-loom-prompt-id"], "b");
assert.equal(a["x-loom-tool-call-id"], "call-a");
assert.equal(b["x-loom-tool-call-id"], "call-b");
assert.equal(b["x-loom-hook-event"], "PostToolUse");
});

test("the shared HTTP transport carries only the event's explicit prompt context", () => {
const url = new URL("../lib/loom-client.mjs", import.meta.url).href;
const script = `
const { readStdinJson, requestJson } = await import(${JSON.stringify(url)});
await readStdinJson();
await requestJson("http://127.0.0.1:1/v1/memory.get", {
body: { memory_key: "example" },
fetchImpl: async (url, options) => {
console.log(JSON.stringify({ headers: options.headers, body: options.body }));
return { status: 200, json: async () => ({}) };
}
});
`;
const env = {
...process.env,
LOOM_CONFIG_FILE: "/nonexistent/loom-config.json",
LOOM_API_KEY_FILE: "/nonexistent/loom-token",
LOOM_API_KEY: "test-secret-must-not-leak",
CLAUDE_PLUGIN_OPTION_TOKEN: "",
};
const capture = (payload) => {
const child = spawnSync(process.execPath, ["--input-type=module", "-e", script], {
input: JSON.stringify(payload),
encoding: "utf8",
timeout: 5000,
env,
});
assert.equal(child.status, 0, child.stderr);
return JSON.parse(child.stdout);
};
const a = capture({ session_id: "c", turn_id: "a", hook_event_name: "UserPromptSubmit", prompt: "private prompt" });
const b = capture({ session_id: "c", turn_id: "b" });
const absent = capture({ session_id: "c" });
const secret = capture({ session_id: "c", turn_id: env.LOOM_API_KEY });
assert.equal(a.headers["x-loom-prompt-id"], "a");
assert.equal(b.headers["x-loom-prompt-id"], "b");
assert.equal(absent.headers["x-loom-prompt-id"], undefined);
assert.equal(secret.headers["x-loom-prompt-id"], undefined);
assert.deepEqual(JSON.parse(a.body), { memory_key: "example" });
assert.equal(JSON.stringify(a).includes("private prompt"), false);
});

test("invocation origin survives a hook event without a host prompt", () => {
const headers = promptHeaders({ session_id: "session-a", hook_event_name: "SessionStart" });
assert.equal(headers["x-loom-call-origin"], "hook");
assert.equal(headers["x-loom-hook-event"], "SessionStart");
assert.equal(headers["x-loom-prompt-id"], undefined);
assert.equal(headers["x-loom-conversation-id"], "session-a");
});

test("detached hooks declare their real entrypoint without inheriting a prompt", async () => {
const { entrypointHeaders } = await import("../lib/prompt-context.mjs");
const headers = entrypointHeaders("/tmp/plugin/hooks/loom-flush.mjs", "/tmp/plugin/hooks");
assert.equal(headers["x-loom-call-origin"], "hook");
assert.equal(headers["x-loom-hook-name"], "loom-flush");
assert.equal(headers["x-loom-prompt-id"], undefined);
assert.deepEqual(entrypointHeaders("/tmp/other/loom-flush.mjs", "/tmp/plugin/hooks"), {});
});
61 changes: 61 additions & 0 deletions integrations/claude-code-plugin/tests/usage.test.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ import { test } from "node:test";
import assert from "node:assert/strict";
import { parseLines } from "../lib/transcript.mjs";
import { extractTurns, recordBody } from "../lib/usage.mjs";
import { extractTurns as extractCodexTurns } from "../../codex-plugin/lib/usage.mjs";

const LINE = (id, extra = {}) =>
JSON.stringify({
Expand Down Expand Up @@ -73,3 +74,63 @@ test("recordBody carries the identity tuple and the consumer", () => {
assert.equal(b.consumer, "claude-code");
assert.equal(b.turns.length, 1);
});

for (const usage of [
{},
{ input_tokens: 9 },
{ output_tokens: 2 },
{ input_tokens: -1, output_tokens: 2 },
{ input_tokens: 1.5, output_tokens: 2 },
{ input_tokens: "9", output_tokens: 2 },
]) {
test(`unmeasured token counters do not create a billing row: ${JSON.stringify(usage)}`, () => {
const { turns, next } = extractTurns([{ type: "assistant", message: { id: "unknown", model: "model", usage } }]);
assert.deepEqual(turns, []);
assert.equal(next.has("unknown"), false);
});
}

test("measured zero token counters remain a recorded row", () => {
const { turns, next } = extractTurns([
{
type: "assistant",
message: { id: "zero", model: "model", usage: { input_tokens: 0, output_tokens: 0 } },
},
]);
assert.equal(turns.length, 1);
assert.equal(turns[0].input_tokens, 0);
assert.equal(turns[0].output_tokens, 0);
assert.equal(next.has("zero"), true);
});

test("an incomplete usage record does not hide a later measured record", () => {
const message = { id: "eventual", model: "model" };
const { turns } = extractTurns([
{ type: "assistant", message: { ...message, usage: {} } },
{ type: "assistant", message: { ...message, usage: { input_tokens: 9, output_tokens: 2 } } },
]);
assert.equal(turns.length, 1);
assert.equal(turns[0].input_tokens, 9);
assert.equal(turns[0].output_tokens, 2);
});

test("Codex usage extraction preserves unknown counts, measured zero and message deduplication", () => {
const row = (id, usage) => ({ type: "assistant", message: { id, model: "model", usage } });
const { turns, next } = extractCodexTurns([
row("unknown", { input_tokens: 9 }),
row("eventual", {}),
row("zero", { input_tokens: 0, output_tokens: 0 }),
row("eventual", { input_tokens: 9, output_tokens: 2 }),
row("eventual", { input_tokens: 9, output_tokens: 2 }),
]);
assert.deepEqual(
turns.map(({ message_id, input_tokens, output_tokens }) => [message_id, input_tokens, output_tokens]),
[
["zero", 0, 0],
["eventual", 9, 2],
],
);
assert.equal(next.has("unknown"), false);
assert.equal(next.size, 2);
assert.deepEqual(extractCodexTurns([row("eventual", { input_tokens: 9, output_tokens: 2 })], next).turns, []);
});
2 changes: 1 addition & 1 deletion integrations/codex-plugin/.codex-plugin/plugin.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "loom",
"version": "1.2.45",
"version": "1.2.48",
"description": "Shared long-term memory and code intelligence for Codex, self-hosted on ClickHouse. MCP tools to save and recall decisions, preferences and corrections across sessions and teammates; lifecycle hooks that capture each session with no model call; a code index that answers where-is, who-calls and blast-radius questions with line-accurate units instead of whole-file reads.",
"author": {
"name": "ClickHouse"
Expand Down
Loading
Loading