feat(models): host-neutral embed/rerank/vision defaults — no default names a machine - #388
Merged
Merged
Conversation
added 9 commits
September 24, 2026 16:55
…ick helper, UI hint)
…nt, smoke scripts, docs
…ults
Provider selection (pickProviderByTask, pickVisionProvider) picks a provider
by task across ANY of its models, but consumers then grabbed models[0] --
wrong when a provider's task-tagged model isn't listed first.
- embeddings.js resolveEmbedConfig / rerank.js resolveRerankConfig: pick the
first model whose task is in EMBED_TASKS/RERANK_TASKS, falling back to
models[0].
- smart-router.js pickVisionProvider: tag the matched image-capable model id
as _preferredModelId on the returned provider; wrapWithVendorLock honors it
over models[0]. Non-vision routes are unaffected.
- embeddings.js resolveEmbedConfig: explicit early guard
("no embedding provider configured"), matching rerank's existing guard.
Tests added (TDD, RED confirmed before the fix): embed-rerank-defaults.test.js
mixed-model-order cases for embed/rerank + the guard-error case;
smart-router.test.js mixed-model-order vision case.
Comment-only: scripts/smoke/smart-router-check.js and local-provider-warmup.js still referenced grackle-vision/grackle-* in comments. docs/architecture/companion.md (+ es mirror): the vision fallback description named "the multimodal 35B" directly instead of the actual resolution order (profile fallback, then crow-chat).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
This is sub-project D1 of the grackle decommission; Kevin is selling grackle. Today every Crow instance embeds through grackle's GPU. That happens because
servers/memory/embeddings.jsfalls back to the hard-coded"grackle-embed",rerank.jsdefaults to"grackle-rerank", and the smart-router's vision route is baked to"grackle-vision". A default that names one machine is a product bug.Measured 2026-09-24: crow already serves the same model, Qwen3-Embedding-0.6B (Q8, 1024 dimensions, last-token pooling), at
100.118.41.122:8004. Embedding the same text on both servers gives cosine 0.999, so switching needs no re-embed.What
servers/shared/provider-task.js(new): a task-based default resolver. Embed resolves in this order:CROW_EMBED_PROVIDERenv;embed_providersetting;embedorembedding;null.Rerank follows the same order with its own env, setting and tags (
rerank/score, since real reranker rows are taggedscore). Results are cached 30 s per task and setting, and the env is checked before the cache. A DB failure givesnull, never a named host.embeddings.js/rerank.js:models[0].nulldegrades to full-text search or no rerank, as a missing provider does today.smart-router.js:DEFAULT_ROUTES.visionistierDefault("vision", null)(the newCROW_SMART_ROUTER_VISIONoverride). Otherwise it picks an image-capable enabled provider, and its image-capable model.embed_providerandrerank_providerstay out of the fleet sync allowlist.No behaviour change at deploy
I read crow, grackle and black-swan's live provider tables read-only. Each has exactly one enabled row tagged embed (
grackle-embed) and one tagged score (grackle-rerank), and no setting. So embed and rerank resolve to the same providers as today until the ops step addscrow-embed. Vision improves: image turns stop going to the deadgrackle-visionrow and reach the multimodalcrow-local35B.Process
docs/superpowers/specs/2026-09-24-host-neutral-model-defaults-design.mddocs/superpowers/plans/2026-09-24-host-neutral-model-defaults.md. Two adversarial review rounds:scoretag, the DB-only rerank rows and a test-helper mismatch;models[0]issue; fixed and re-reviewed.After merge (ops, not in this PR)
crow-embedprovider row on crow (it syncs) and on r4, and setembed_provider=crow-embed.CROW_EMBED_PROVIDER/EMBED_HOST.grackle-rerankandgrackle-visionrows.grackle-embedretires with grackle.