Skip to content

feat(models): host-neutral embed/rerank/vision defaults — no default names a machine - #388

Merged
kh0pper merged 9 commits into
mainfrom
feat/host-neutral-model-defaults
Sep 24, 2026
Merged

kh0pper merged 9 commits into
mainfrom
feat/host-neutral-model-defaults

Conversation

@kh0pper

@kh0pper kh0pper commented Sep 24, 2026

Copy link
Copy Markdown
Owner

Why

This is sub-project D1 of the grackle decommission; Kevin is selling grackle. Today every Crow instance embeds through grackle's GPU. That happens because servers/memory/embeddings.js falls back to the hard-coded "grackle-embed", rerank.js defaults to "grackle-rerank", and the smart-router's vision route is baked to "grackle-vision". A default that names one machine is a product bug.

Measured 2026-09-24: crow already serves the same model, Qwen3-Embedding-0.6B (Q8, 1024 dimensions, last-token pooling), at 100.118.41.122:8004. Embedding the same text on both servers gives cosine 0.999, so switching needs no re-embed.

What

  • servers/shared/provider-task.js (new): a task-based default resolver. Embed resolves in this order:
    1. CROW_EMBED_PROVIDER env;
    2. the embed_provider setting;
    3. the lowest-id enabled provider that has a model tagged embed or embedding;
    4. otherwise null.
      Rerank follows the same order with its own env, setting and tags (rerank/score, since real reranker rows are tagged score). Results are cached 30 s per task and setting, and the env is checked before the cache. A DB failure gives null, never a named host.
  • embeddings.js / rerank.js:
    • They use the resolver, and use the matching model rather than models[0].
    • rerank now also reads rows that exist only in the DB.
    • null degrades to full-text search or no rerank, as a missing provider does today.
  • smart-router.js: DEFAULT_ROUTES.vision is tierDefault("vision", null) (the new CROW_SMART_ROUTER_VISION override). Otherwise it picks an image-capable enabled provider, and its image-capable model.
  • Also updated: the Settings AI-profiles hint, the four manual smoke scripts (now env-driven and host-neutral), and the docs (EN + ES).
  • Unchanged: embed_provider and rerank_provider stay out of the fleet sync allowlist.

No behaviour change at deploy

I read crow, grackle and black-swan's live provider tables read-only. Each has exactly one enabled row tagged embed (grackle-embed) and one tagged score (grackle-rerank), and no setting. So embed and rerank resolve to the same providers as today until the ops step adds crow-embed. Vision improves: image turns stop going to the dead grackle-vision row and reach the multimodal crow-local 35B.

Process

  • Spec: docs/superpowers/specs/2026-09-24-host-neutral-model-defaults-design.md
  • Plan: docs/superpowers/plans/2026-09-24-host-neutral-model-defaults.md. Two adversarial review rounds:
    • round 1 found the score tag, the DB-only rerank rows and a test-helper mismatch;
    • round 2 approved.
  • Build: 3 subagent-driven tasks, each reviewed.
  • Final review: Ready with fixes. It found the any-model-match vs models[0] issue; fixed and re-reviewed.
  • Tests: full suite 5147 pass / 0 fail (Node 24).

After merge (ops, not in this PR)

  1. Add a crow-embed provider row on crow (it syncs) and on r4, and set embed_provider=crow-embed.
  2. Repoint r4's CROW_EMBED_PROVIDER / EMBED_HOST.
  3. Disable the dead grackle-rerank and grackle-vision rows.
  4. grackle-embed retires with grackle.

kh0pper added 9 commits September 24, 2026 16:55
…ults

Provider selection (pickProviderByTask, pickVisionProvider) picks a provider
by task across ANY of its models, but consumers then grabbed models[0] --
wrong when a provider's task-tagged model isn't listed first.

- embeddings.js resolveEmbedConfig / rerank.js resolveRerankConfig: pick the
  first model whose task is in EMBED_TASKS/RERANK_TASKS, falling back to
  models[0].
- smart-router.js pickVisionProvider: tag the matched image-capable model id
  as _preferredModelId on the returned provider; wrapWithVendorLock honors it
  over models[0]. Non-vision routes are unaffected.
- embeddings.js resolveEmbedConfig: explicit early guard
  ("no embedding provider configured"), matching rerank's existing guard.

Tests added (TDD, RED confirmed before the fix): embed-rerank-defaults.test.js
mixed-model-order cases for embed/rerank + the guard-error case;
smart-router.test.js mixed-model-order vision case.
Comment-only: scripts/smoke/smart-router-check.js and
local-provider-warmup.js still referenced grackle-vision/grackle-* in
comments. docs/architecture/companion.md (+ es mirror): the vision
fallback description named "the multimodal 35B" directly instead of the
actual resolution order (profile fallback, then crow-chat).
@kh0pper
kh0pper merged commit a90d901 into main Sep 24, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant