Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -133,7 +133,7 @@ LOOM_EMBEDDING_MODEL=fake-ngram-3072
LOOM_EMBEDDING_DIM=3072
# LLM: fake, openai or anthropic. Empty (code default) = inferred from the model
# name, else from the key present, else fake. Empty model = the provider's
# default (gpt-6-luna / claude-haiku-5-5 / fake-echo-1).
# default (gpt-6-luna / claude-sonnet-5-5 / fake-echo-1).
LOOM_LLM_PROVIDER=fake
LOOM_LLM_MODEL=fake-echo-1
# Provider keys; each is needed only by the provider that names it.
Expand Down
3 changes: 2 additions & 1 deletion docs/architecture/llm-architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -179,7 +179,8 @@ low.
2. Else from `LOOM_LLM_MODEL`: `bedrock:*` is `bedrock`; `claude*` is `anthropic`; `gpt*`, `o1*`, `o3*`, `o4*`,
`chatgpt*` are `openai`; `fake*` is `fake`.
3. Else from the key present: `OPENAI_API_KEY` gives `openai` with `gpt-6-luna`, then
`ANTHROPIC_API_KEY` gives `anthropic` with `claude-haiku-5-5`.
`ANTHROPIC_API_KEY` gives `anthropic` with `claude-sonnet-5-5`.
An explicit deployment model or per-job model overrides this provider default.
4. Else `fake` (`fake-echo-1`).

`Settings` resolves it once at load; `settings.llm_provider` is always concrete, and
Expand Down
2 changes: 1 addition & 1 deletion docs/getting-started/models-and-keys.md
Original file line number Diff line number Diff line change
Expand Up @@ -127,7 +127,7 @@ names its providers explicitly (the local embedder and the fake LLM).

```dotenv
LOOM_LLM_PROVIDER=anthropic
LOOM_LLM_MODEL=claude-haiku-5-5
LOOM_LLM_MODEL=claude-sonnet-5-5
ANTHROPIC_API_KEY=<your Anthropic API key>
LOOM_REQUIRE_REAL_PROVIDERS=true
```
Expand Down
2 changes: 1 addition & 1 deletion docs/operations/deployment.md
Original file line number Diff line number Diff line change
Expand Up @@ -669,7 +669,7 @@ conflicts and writes `memory.ask` answers. Both default to fakes that need no ne
| ------------------------- | -------------------------------------------------------------- | ------------------------ |
| `fake` | nothing; extraction produces nothing | `fake-echo-1` |
| `openai` | `OPENAI_API_KEY` | `gpt-6-luna` |
| `anthropic` | `ANTHROPIC_API_KEY` | `claude-haiku-5-5` |
| `anthropic` | `ANTHROPIC_API_KEY` | `claude-sonnet-5-5` |
| `local` | a chat-completions server at `LOOM_LOCAL_LLM_BASE_URL`, no key | `qwen2.5:3b` |

- **Width.** `LOOM_EMBEDDING_DIM` must equal the model's width. It is fixed when the
Expand Down
2 changes: 1 addition & 1 deletion docs/reference/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -1577,7 +1577,7 @@ Models: which model does each job.

| Variable | Type | Default | Live | Meaning |
| --- | --- | --- | --- | --- |
| `LOOM_LLM_MODEL` | `str` | `''` | | The deployment's LLM: memory extraction, rerank and every job whose own model setting (the other *_model settings) is left "". Empty takes the resolved provider's default model (gpt-6-luna / claude-haiku-5-5 / qwen2.5:3b / fake-echo-1). |
| `LOOM_LLM_MODEL` | `str` | `''` | | The deployment's LLM: memory extraction, rerank and every job whose own model setting (the other *_model settings) is left "". Empty takes the resolved provider's default model (gpt-6-luna / claude-sonnet-5-5 / qwen2.5:3b / fake-echo-1). |
| `LOOM_ASSISTANT_MODEL` | `str` | `''` | | The assistant agent's model, which Loom writes into the agent definition at boot. "" = the definition's own (integrations/librechat/agent.yaml), not llm_model. |
| `LOOM_ASSISTANT_PROVIDER` | `'' \| 'anthropic' \| 'local'` | `''` | | The LibreChat endpoint the agent runs on: "" = the definition's (anthropic), "local" = LibreChat's local endpoint (a model server the operator runs; needs assistant_model). |
| `LOOM_LLM_FALLBACK_MODEL` | `str` | `''` | | Model to answer with when a client's provider is still unavailable after its retries (429 / 5xx / transport / timeout) or rejected the request (400 / 413 / 422). Counted in `loom.llm.fallback`. 401 / 403 / 404 are configuration and never fall back. "" = fail the op. |
Expand Down
2 changes: 1 addition & 1 deletion loom.example.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -268,7 +268,7 @@ settings:

# The deployment's LLM: memory extraction, rerank and every job whose own model
# setting (the other *_model settings) is left "". Empty takes the resolved provider's
# default model (gpt-6-luna / claude-haiku-5-5 / qwen2.5:3b / fake-echo-1). [restart]
# default model (gpt-6-luna / claude-sonnet-5-5 / qwen2.5:3b / fake-echo-1). [restart]
# llm_model: ""

# The assistant agent's model, which Loom writes into the agent definition at boot. ""
Expand Down
2 changes: 1 addition & 1 deletion schemas/loom-settings.schema.json
Original file line number Diff line number Diff line change
Expand Up @@ -3690,7 +3690,7 @@
"llm_model": {
"default": "",
"type": "string",
"description": "The deployment's LLM: memory extraction, rerank and every job whose own model setting (the other *_model settings) is left \"\". Empty takes the resolved provider's default model (gpt-6-luna / claude-haiku-5-5 / qwen2.5:3b / fake-echo-1)."
"description": "The deployment's LLM: memory extraction, rerank and every job whose own model setting (the other *_model settings) is left \"\". Empty takes the resolved provider's default model (gpt-6-luna / claude-sonnet-5-5 / qwen2.5:3b / fake-echo-1)."
},
"llm_openai_reasoning_effort": {
"default": "",
Expand Down
6 changes: 3 additions & 3 deletions server/core/llm_provider.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,13 +13,13 @@
# The API-key environment variable of each keyed provider.
KEY_ENV_BY_PROVIDER = {"openai": "OPENAI_API_KEY", "anthropic": "ANTHROPIC_API_KEY"}

# The model each provider runs when LOOM_LLM_MODEL is unset: the cheapest current
# model of each provider that does the extraction job well.
# The model each provider runs when LOOM_LLM_MODEL is unset.
# Explicit deployment and per-job model selections keep their configured values.
DEFAULT_MODEL_BY_PROVIDER = {
# Bedrock model access and inference profiles belong to the deployment.
"bedrock": "",
"openai": "gpt-6-luna",
"anthropic": "claude-haiku-5-5",
"anthropic": "claude-sonnet-5-5",
# A server the operator runs (`local`): a 3B model Ollama pulls in about 2 GB.
"local": "qwen2.5:3b",
"fake": "fake-echo-1",
Expand Down
2 changes: 1 addition & 1 deletion server/settings/catalog.py
Original file line number Diff line number Diff line change
Expand Up @@ -4099,7 +4099,7 @@ class ModelSettings(BaseModel):
# --- defaults ---------------------------------------------------------------
# The deployment's LLM: memory extraction, rerank and every job whose own model
# setting (the other *_model settings) is left "". Empty takes the resolved
# provider's default model (gpt-6-luna / claude-haiku-5-5 / qwen2.5:3b / fake-echo-1).
# provider's default model (gpt-6-luna / claude-sonnet-5-5 / qwen2.5:3b / fake-echo-1).
llm_model: str = ""
# The assistant agent's model, which Loom writes into the agent definition at boot.
# "" = the definition's own (integrations/librechat/agent.yaml), not llm_model.
Expand Down
10 changes: 8 additions & 2 deletions tests/unit/test_llm_provider_inference.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,13 @@
[
# Explicit provider always wins, model defaults per provider.
("openai", "", NONE, ResolvedLLM("openai", "gpt-6-luna", "LOOM_LLM_PROVIDER")),
("anthropic", "", NONE, ResolvedLLM("anthropic", "claude-haiku-5-5", "LOOM_LLM_PROVIDER")),
("anthropic", "", NONE, ResolvedLLM("anthropic", "claude-sonnet-5-5", "LOOM_LLM_PROVIDER")),
(
"anthropic",
"claude-haiku-5-5",
ANTHROPIC,
ResolvedLLM("anthropic", "claude-haiku-5-5", "LOOM_LLM_PROVIDER"),
),
("fake", "", BOTH, ResolvedLLM("fake", "fake-echo-1", "LOOM_LLM_PROVIDER")),
("fake", "gpt-4o-mini", OPENAI, ResolvedLLM("fake", "gpt-4o-mini", "LOOM_LLM_PROVIDER")),
(" OpenAI ", "gpt-5", NONE, ResolvedLLM("openai", "gpt-5", "LOOM_LLM_PROVIDER")),
Expand Down Expand Up @@ -54,7 +60,7 @@
"",
"",
ANTHROPIC,
ResolvedLLM("anthropic", "claude-haiku-5-5", "ANTHROPIC_API_KEY is set"),
ResolvedLLM("anthropic", "claude-sonnet-5-5", "ANTHROPIC_API_KEY is set"),
),
("", "", BOTH, ResolvedLLM("openai", "gpt-6-luna", "OPENAI_API_KEY is set")),
# Nothing at all: the keyless fake default the dev container relies on.
Expand Down