Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
53 commits
Select commit Hold shift + click to select a range
30c73a6
Document agent platform integration and repository evaluation
zlareb1 Nov 20, 2026
8bfd16c
Clarify cached answers and repository listing metadata
zlareb1 Nov 20, 2026
1e7cd02
Compose current main with agent documentation
zlareb1 Nov 20, 2026
4db1211
Align code response descriptions with retrieval and listing behavior
zlareb1 Nov 20, 2026
d667581
Clarify provider setup and failure diagnostics
zlareb1 Nov 20, 2026
0a04755
Compose documentation answers with current history guidance
zlareb1 Nov 20, 2026
f93385b
Route the conflict judge in the single-provider recipe
zlareb1 Nov 20, 2026
2fbcd5f
Compose provider guidance with current history documentation
zlareb1 Nov 20, 2026
a6d8a12
Describe the dashboard model selection boundary
zlareb1 Nov 20, 2026
fc3a5f7
Compose provider instructions with current cost guidance
zlareb1 Nov 20, 2026
0a0e264
Document agent integration and provider setup
zlareb1 Nov 20, 2026
54d4fd5
Compose answerability guides with the replicated fixture repair
zlareb1 Nov 20, 2026
1f1e2bc
Compose answerability guides with current CI and fixture checks
zlareb1 Nov 20, 2026
6090dda
Compose current main with agent documentation
zlareb1 Nov 21, 2026
165173f
Compose repository documentation fixes with agent guides
zlareb1 Nov 21, 2026
e9dedfd
Compose current documentation sources
zlareb1 Nov 21, 2026
fa087d3
Compose current integration documentation
zlareb1 Nov 21, 2026
e5ba08a
Compose main with agent answerability docs
zlareb1 Nov 21, 2026
0d8ae1d
Compose current main with agent answerability docs
zlareb1 Nov 21, 2026
da15d6a
Compose current main with agent answerability docs
zlareb1 Nov 21, 2026
9693197
Compose agent documentation with main
zlareb1 Nov 21, 2026
c45642b
Compose agent documentation with main
zlareb1 Nov 21, 2026
420873f
Compose agent documentation with main
zlareb1 Nov 21, 2026
92dfaa1
Compose agent documentation with main
zlareb1 Nov 21, 2026
bf86f4a
Merge current main into answerability docs
zlareb1 Nov 21, 2026
7829d1c
Document provider caches across configuration changes
zlareb1 Nov 21, 2026
e4c455f
Document provider costs and access prerequisites accurately
zlareb1 Nov 21, 2026
a53e5c0
Merge current main into agent answerability documentation
zlareb1 Nov 21, 2026
f728121
Document model selection and accounting limits
zlareb1 Nov 21, 2026
0efcd74
Document extraction outside daily cost caps
zlareb1 Nov 21, 2026
ae41638
Merge main into documentation answerability
zlareb1 Nov 21, 2026
05b962c
Merge main into the agent documentation update
zlareb1 Nov 21, 2026
8e8d1f8
Clarify repository access overrides and answer cache checks
zlareb1 Nov 21, 2026
a23fe90
Document provider calls and optional sandbox boundaries
zlareb1 Nov 21, 2026
4a34863
Merge main for native worker diagnostics
zlareb1 Nov 21, 2026
deb9994
Merge current main into agent answerability documentation
zlareb1 Nov 21, 2026
9a122c7
Merge current main into agent answerability documentation
zlareb1 Nov 21, 2026
798aed6
Merge current main for documentation checks
zlareb1 Nov 21, 2026
2758853
Qualify budget failures by caller
zlareb1 Nov 21, 2026
bcc7762
Merge current main for documentation checks
zlareb1 Nov 21, 2026
233a421
Merge main into agent answerability documentation
zlareb1 Nov 21, 2026
4af5471
Describe verifier failures and embedding publication
zlareb1 Nov 21, 2026
65e54e1
Merge main with Python compiler scope fixes
zlareb1 Nov 21, 2026
28adedb
Compose current provider defaults with agent documentation
zlareb1 Nov 21, 2026
9ff998e
Document erased-index recovery and asynchronous link completion
zlareb1 Nov 21, 2026
3d86c65
Merge current main into docs answerability
zlareb1 Nov 22, 2026
181e830
Clarify indexing, provider and retirement boundaries
zlareb1 Nov 22, 2026
1305c64
Merge current main into agent answerability documentation
zlareb1 Nov 22, 2026
1eee27d
Regenerate repository listing description
zlareb1 Nov 22, 2026
8ca2143
Clarify provider probes and compiler indexing boundaries
zlareb1 Nov 22, 2026
d19c58f
Scope cost limits and evaluation organization explicitly
zlareb1 Nov 22, 2026
7be0ae2
Merge main into agent answerability documentation
zlareb1 Nov 22, 2026
bf1ccc8
Point code access denials to token and grant rules
zlareb1 Nov 22, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
150 changes: 90 additions & 60 deletions docs/architecture/llm-architecture.md

Large diffs are not rendered by default.

34 changes: 19 additions & 15 deletions docs/features/code-index/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,8 @@ the index is built is in
The [language support table](#language-support) distinguishes fast-tier symbols and
edges from compiler-tier coverage.

The code workers are off by default; a deployment turns them on per repository
Worker defaults vary by tier; check the [workers reference](../../reference/workers.md).
A worker also needs an admitted supervisor and a repository configured for its tier
([repositories](repositories.md)). The default `core` MCP toolset lists no code tools
unless `LOOM_MCP_CODE_TOOLS=true`; `?toolset=coding` always does.

Expand Down Expand Up @@ -87,8 +88,11 @@ Structure is extracted twice. Each symbol and edge names its tier (`tier`: `scip
[code index operations](../../operations/code-index-operations.md#compiler-tier)). An
external producer can upload with `code.scip_ingest`. It gives exact call, override
and inheritance edges. Its C/C++, Go and TypeScript steps execute the repository's own
build code; they run inside the
[repository-job sandbox](../../operations/code-index-operations.md#repository-job-sandbox).
build code. They use the
[repository-job sandbox](../../operations/code-index-operations.md#repository-job-sandbox)
when available. With `LOOM_CODE_SANDBOX_REQUIRED=true` they refuse to run without
it; otherwise, operator-configured repositories can run those builds as plain
processes.

Why two: the fast tier is cheap, so a merge is searchable after one ingest pass. The
compiler tier needs a build environment and trails. Serving both, labelled, gives an
Expand Down Expand Up @@ -317,18 +321,18 @@ generated from the same schemas.
Start with `code.index_status` (watermarks, lags, tiers, worker errors) and
`code.coverage` (what the namespace holds).

| Symptom | Cause | Fix |
| ------------------------------------------ | --------------------------------------------------------- | --------------------------------------------------------------------- |
| a symbol you just added is not found | uncommitted, unpushed, or not yet ingested | grep your tree; `code.index_status` shows the lag |
| `lag_commits: 0` but merges are missing | the mirror stopped fetching | `clone_fetched_at` / `fetch_overdue`, then `/v1/admin/workers.status` |
| a fix you know exists is not found | wrong branch namespace | `code.grep mode=presence namespaces=[…]` |
| `AUTH_INSUFFICIENT` on every `code.*` call | the token does not name the code namespace | join the repository, or mint with `--namespaces` |
| a call that names no namespace is refused | no `X-Loom-Code-Scope` header and no deployment default | pass `namespace`, or set `LOOM_CODE_ORG` / `LOOM_CODE_NAMESPACE` |
| no code tools in the MCP list | the `core` toolset without `LOOM_MCP_CODE_TOOLS=true` | mount `?toolset=coding`, or set the flag |
| `code.search` says lexical-only | embedding coverage below `LOOM_CODE_READY_EMBED_COVERAGE` | enable `code_embed_recovery`; watch `embedding_coverage` |
| callers look incomplete for a language | the language has only the fast tier | read `epistemic` / `causes`; `code.coverage` shows the `scip` column |
| every namespace re-indexing after a deploy | an extractor module changed (the stamp moved) | expected; progress is in `code_index_state` |
| a self-service repository stuck in `error` | clone or first index failed | read `stage_detail` on `code.repos` |
| Symptom | Cause | Fix |
| ------------------------------------------ | --------------------------------------------------------- | ------------------------------------------------------------------------------------------ |
| a symbol you just added is not found | uncommitted, unpushed, or not yet ingested | grep your tree; `code.index_status` shows the lag |
| `lag_commits: 0` but merges are missing | the mirror stopped fetching | `clone_fetched_at` / `fetch_overdue`, then `/v1/admin/workers.status` |
| a fix you know exists is not found | wrong branch namespace | `code.grep mode=presence namespaces=[…]` |
| `AUTH_INSUFFICIENT` on every `code.*` call | the caller lacks sufficient access for the operation | check token scope and live grants against [the access rules](repositories.md#who-can-read) |
| a call that names no namespace is refused | no `X-Loom-Code-Scope` header and no deployment default | pass `namespace`, or set `LOOM_CODE_ORG` / `LOOM_CODE_NAMESPACE` |
| no code tools in the MCP list | the `core` toolset without `LOOM_MCP_CODE_TOOLS=true` | mount `?toolset=coding`, or set the flag |
| `code.search` says lexical-only | embedding coverage below `LOOM_CODE_READY_EMBED_COVERAGE` | enable `code_embed_recovery`; watch `embedding_coverage` |
| callers look incomplete for a language | the language has only the fast tier | read `epistemic` / `causes`; `code.coverage` shows the `scip` column |
| every namespace re-indexing after a deploy | an extractor module changed (the stamp moved) | expected; progress is in `code_index_state` |
| a self-service repository stuck in `error` | clone or first index failed | read `stage_detail` on `code.repos` |

Operator signals (tier lag, stale files, failed passes) are in
[code index operations](../../operations/code-index-operations.md).
Expand Down
94 changes: 79 additions & 15 deletions docs/features/code-index/repositories.md
Original file line number Diff line number Diff line change
Expand Up @@ -136,9 +136,11 @@ Rules:
- A private repository's token is stored encrypted with a key derived from
`LOOM_JWT_SECRET` and never returned. Rotating that secret makes stored tokens
unusable until a maintainer enters them again.
- `code.repo.remove` retires the registry rows and purges every `code_*` row of the
namespace through a resumable erasure saga. The mirror is deleted when no other branch
uses it. An operator-configured entry is refused with `REPO_FROM_CONFIG`.
- `code.repo.remove` makes the registry rows inactive and starts a resumable namespace
erasure saga. It purges indexed code and search telemetry; cached answers follow the
[retention rules](../../operations/code-index-operations.md#rollout-and-shutdown).
The mirror is deleted when no other branch uses it. An operator-configured entry is
refused with `REPO_FROM_CONFIG`.
- `LOOM_CODE_SELF_SERVICE_ENABLED=false` removes the feature (`SELF_SERVICE_OFF`).
- `LOOM_CODE_SELF_SERVICE_PLATFORM_ADMIN_ONLY=true` leaves `code.repo.remote_branches`,
`code.repo.add` and `code.repo.add_branch` to platform admins, beyond the role each op
Expand Down Expand Up @@ -177,12 +179,12 @@ or SQL.
`compiler_candidates` is inferred from build manifests. A candidate without a watermark
appears in `quality_warnings` with the workers to enable:

| Candidate | Preflight in the worker environment | Worker |
| --------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------- |
| C/C++ | `scip-clang --version` (in the amd64 image); the repository's CMake configures with clang 21 | `code_scip_full`, then `code_scip_delta` |
| TypeScript/JavaScript | Indexer and package managers included in Docker; native servers run `sh scripts/install-code-toolchains.sh`. Repository dependencies are prepared automatically. | `code_scip_ts_delta` |
| Go | `scip-go --version`; `go env` succeeds for every module | `code_scip_go_delta` |
| Python | `scip-python --version`; project imports resolve in its environment | `code_scip_py_delta`, then `code_scip_apply` |
| Candidate | Preflight in the worker environment | Worker |
| --------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------- |
| C/C++ | `scip-clang --version` (in the amd64 image); the repository's CMake configures with clang 21 | `code_scip_full`, then `code_scip_delta` |
| TypeScript/JavaScript | Indexer and package managers included in Docker; native servers run `sh scripts/install-code-toolchains.sh`. Repository dependencies are prepared automatically. | `code_scip_ts_delta` |
| Go | `scip-go --version`; `go env` succeeds for every module | `code_scip_go_delta` |
| Python | `scip-python --version`; project imports resolve in its environment | `code_scip_py_delta` |

The fast tier remains usable when compiler preflight fails. `code.index_status` and the
Health tab expose worker errors, per-language coverage, unresolved edges, embedding
Expand All @@ -204,18 +206,80 @@ until `loom code backfill` runs. Every setting is in the

Code namespaces follow the usual grant rules
([identities and namespaces](../../concepts/identities-and-namespaces.md),
[tokens and grants](../../guides/tokens-and-grants.md)): a token must name the code
namespace (or `*`), or the call fails with `AUTH_INSUFFICIENT`. Three switches change
that:
[tokens and grants](../../guides/tokens-and-grants.md)). A standard code namespace
accepts a sufficient token claim or a live reader grant for the caller's org;
existing grant revocations and role caps still apply. Three switches change access:

| Switch | Effect |
| ----------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------- |
| `LOOM_CODE_ORG_READABLE=true`, or the entry's `org_readable` | every authenticated principal of the org reads without a grant (default off); writes stay grant-driven |
| `LOOM_CODE_ORG_READABLE=true`, or the entry's `org_readable` | authenticated org members read standard namespaces without a grant (default off); write access is unchanged |
| `LOOM_CODE_REPO_PUBLIC=true`, or the entry's `public` | the namespace registers `join_policy=open` and discoverable, so any org member joins as a reader at once; otherwise the policy is `approval` |
| `classification: "restricted"` (entry, or `code.repo.set_access`) | never org-readable or public; a maintainer approves each reader, and every read needs a token naming the namespace plus a live direct grant |

A `code.repo.set_access` row wins over the configured classification and join policy
and survives restarts. `code.refresh` (writer) and the admin ops always need a grant.
and survives restarts. `code.refresh` requires `writer`; admin operations require
`admin`. For standard namespaces, token claims or grants can supply those roles,
subject to scoped-token ceilings and live grant caps. Restricted namespaces require
both the token's own role and a live direct grant, with literal namespace naming.

### Separate repositories for two teams

Keep repositories with different readers in different code namespaces. For example,
index a billing repository as `code-billing` and an operations repository as
`code-operations`, both `classification: "restricted"` with `join_policy: "approval"`.
Use the [operator-configured entry format](../../operations/code-index-operations.md)
or the [self-service access controls](#self-service-repositories) for your deployment.
Bootstrap maintainer grants before restricting an existing namespace, following the
[restricted-namespace procedure](../../guides/tokens-and-grants.md#restricted-namespaces-classificationrestricted).

Give a billing service a live viewer grant and a token explicitly naming only
`code-billing`. Its expected read behavior is:

| Request | Expected scope outcome |
| ---------------------------------- | ------------------------------------------------------------------------------ |
| `code.search` in `code-billing` | Authorized; useful hits still depend on the index and query |
| `code.search` in `code-operations` | Denied; the billing grant and token do not cover it |
| `code.repos` | Can include request-only metadata for the other repository; inspect `readable` |

Check the allowed request with a symbol you know is indexed, then repeat an equivalent
request against the other namespace. Over HTTP, a denied scoped code read returns
`AUTH_INSUFFICIENT`; a known object ID can instead read as missing to avoid revealing its
existence ([authorization diagnostics](../../operations/troubleshooting.md#an-id-you-know-exists-reads-as-missing)).
Test revocation separately from an empty search result. Repeat for each indexed branch:
namespace naming and maintainer inheritance do not give every reader access to every
branch.

A same-organization repository with approval-based joining can appear in `code.repos`
as `readable: false`. That row exposes metadata for requesting access, with empty index
counts and watermarks. It does not authorize reading code. Do not treat a listing entry
as a successful access check.

For example, with the billing service's token, replace the query with a known billing
symbol. Repeat with `namespace` set to `code-operations` and expect an authorization
failure, not an empty successful result:

```sh
curl --fail-with-body -X POST "$LOOM_BASE_URL/v1/code.search" \
-H "Authorization: Bearer $LOOM_API_KEY" \
-H 'Content-Type: application/json' \
--data '{"org":"example","namespace":"code-billing","query":"invoice authorization","top_k":3}'
```

The Git token or SSH identity that fetches source is separate from the Loom token and
grants used by readers. A chat or document connector login does not grant a native Loom
service repository access. Agent names and repository tags record attribution or
association; they are not access boundaries.

### Check indexing isolation separately from read access

Repository read grants do not sandbox indexing jobs. Self-service indexing requires
the sandbox unless an operator explicitly enables its restricted unsandboxed mode;
operator-configured repositories can run without it. To require isolation for every
repository job, set `LOOM_CODE_SANDBOX_REQUIRED=true` and satisfy the host requirements.
When the sandbox is unavailable, jobs are refused or skipped rather than silently run
outside it. The [sandbox runbook](../../operations/code-index-operations.md#repository-job-sandbox)
describes namespaces, mounts, egress, process limits, supported stacks and remaining
container/host responsibilities; Loom's per-process limits do not impose a job cgroup.

## Failure modes

Expand All @@ -226,7 +290,7 @@ and survives restarts. `code.refresh` (writer) and the admin ops always need a g
| Adding a repository answers `AUTH_INSUFFICIENT` | the message names `LOOM_CODE_SELF_SERVICE_PLATFORM_ADMIN_ONLY`: only platform admins add repositories and branches on this deployment, with a token of their own that is not scoped to namespaces |
| A self-service card stays at `error` | read `stage_detail` in `code.repos`; a failed clone retries next pass (timeout `LOOM_CODE_BOOTSTRAP_CLONE_TIMEOUT_S`) |
| A private self-service repository stops fetching | `LOOM_JWT_SECRET` was rotated; re-enter the token |
| An entry is missing from `code.repos` | check `loom.code.repos_config_errors`; `code.repos` also lists only what the caller may read |
| An entry is missing from `code.repos` | check `loom.code.repos_config_errors` and the caller's listing eligibility; request-only entries have `readable: false` and do not grant code access |
| Revoking or lowering an admin answers busy | a revoke on a code namespace first re-reads `loom.code_repos`, so it sees the branches added here from another process, and answers busy while that read fails; retry once the read succeeds |
| A restricted branch added here has no admin | its admins were all copies from branches added here, which the sync revokes; grant admin on one of the repository's operator entries (with none, the operator lists a branch of it in `LOOM_CODE_REPOS_JSON` first) |
| A new maintainer lacks the other branches | the admin grant was made before this process read `loom.code_repos` (that process logs `not synced` the first time); the next admin grant or approval on a branch of the repository, or a branch added, copies it; or grant it again |
Expand Down
Loading
Loading