Skip to content

Honour EMBEDDER_PROVIDER, and allow model/dimensions from the environment - #65

Open
a-optic wants to merge 1 commit into
benclawbot:masterfrom
a-optic:feat/env-configurable-embedder
Open

Honour EMBEDDER_PROVIDER, and allow model/dimensions from the environment#65
a-optic wants to merge 1 commit into
benclawbot:masterfrom
a-optic:feat/env-configurable-embedder

Conversation

@a-optic

@a-optic a-optic commented Sep 9, 2026

Copy link
Copy Markdown

A documented variable that does nothing

.env.example ships EMBEDDER_PROVIDER=ollama and tells the reader to "set EMBEDDER_PROVIDER and provide the matching API key below". EmbedderConfig only ever reads provider from config/settings.yaml, so setting the documented variable has no effect.

The harder half: model and dimensions

model and dimensions are config-file-only too, and in a container deployment config/settings.yaml is baked into the image at build time. Running a different embedding model therefore means editing a tracked file and rebuilding — and that file is then the one most likely to conflict on every upgrade, for anyone not using the default model.

The change

All three now follow the same env-over-config precedence max_chars and chunk_overlap already use:

EMBEDDER_PROVIDER    # previously documented but inert
EMBEDDER_MODEL       # new
EMBEDDER_DIMENSIONS  # new

Two details worth calling out:

  • A set-but-empty variable is treated as unset rather than as a configured empty string. .env files routinely carry blank placeholders, and letting one through would silently select no provider at all.
  • dimensions is validated as a positive integer, matching the existing max_chars / chunk_overlap checks.

.env.example documents the two new variables, including the constraint that they must agree with the width of the embedding column.

Verification

Against a running deployment:

no env set                       -> settings.yaml wins   (arctic-embed2, 1024)
EMBEDDER_MODEL / _DIMENSIONS set -> env wins             (mxbai-embed-large, 512)
EMBEDDER_MODEL=""   or "   "     -> falls back to settings.yaml
EMBEDDER_DIMENSIONS=0            -> ValueError

Scope

This does not change where the SQL dimension comes from — schema.sql and migration 015_embedding_dim_change.sql still hardcode 768, so a deployment on a different width still manages the column itself. Happy to follow up on driving that from the same setting if you'd want it.

…ironment

.env.example documents EMBEDDER_PROVIDER and tells the reader to "set
EMBEDDER_PROVIDER and provide the matching API key below", but
EmbedderConfig only ever read `provider` from config/settings.yaml. Setting
the documented variable had no effect.

The same is true of `model` and `dimensions`, which is the harder problem in
a container deployment: settings.yaml is baked into the image at build time,
so running a different embedding model means editing a tracked file and
rebuilding. That is also the file most likely to conflict on every upgrade,
for anyone not using the default model.

All three now follow the env-over-config precedence that max_chars and
chunk_overlap already use, so a deployment can select its model and vector
width without touching a tracked file.

A set-but-empty variable is treated as unset rather than as a configured
empty string: .env files routinely carry blank placeholders, and letting one
through would silently select no provider at all.

dimensions is validated as a positive integer, matching the existing
max_chars and chunk_overlap checks.

Verified against a running deployment:

  no env set                      -> settings.yaml wins  (arctic-embed2, 1024)
  EMBEDDER_MODEL/-DIMENSIONS set  -> env wins            (mxbai-embed-large, 512)
  EMBEDDER_MODEL=""  or "   "     -> falls back to settings.yaml
  EMBEDDER_DIMENSIONS=0           -> ValueError

Note this does not change where the SQL dimension comes from: schema.sql and
migration 015 still hardcode 768, so a deployment using a different width
still has to manage the column itself. Happy to follow up on that separately
if you'd like it driven from the same setting.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant