Skip to content

feat: add Ollama metadata enrichment (context length + capabilities via /api/show) - #163

Open
DjFabFab wants to merge 9 commits into
yuhp:devfrom
DjFabFab:feat/ollama-model-info-enrichment
Open

DjFabFab wants to merge 9 commits into
yuhp:devfrom
DjFabFab:feat/ollama-model-info-enrichment

Conversation

@DjFabFab

@DjFabFab DjFabFab commented Oct 18, 2026 •

Copy link
Copy Markdown

Title: feat: add Ollama metadata enrichment (context length + capabilities via /api/show)

Body:

Problem

Ollama's OpenAI-compatible GET /v1/models returns minimal objects ({ id, object, created, owned_by }), so Ollama providers get no enrichment: V1 injects no limit field; V2 blindly falls back to DEFAULT_CONTEXT_TOKEN_LIMIT (200k/32k). The native POST /api/show endpoint does expose per-model context length and capabilities but is unused.

Solution

Adds an explicit, opt-in modelInfoFormat: "ollama" metadata enricher following the existing LM Studio data-style pattern (fetch once at resolution time, sync Map lookup in enrich()):

  • src/types/plugin-config.ts — ModelInfoFormat.Ollama = 'ollama'.
  • src/utils/model-info/ollama.ts (new) — createOllamaEnricher(data): Map keyed by model id → parsed /api/show body. Context resolution order: model_info.<details.family>.context_length → any *.context_length key → top-level context_length → num_ctx parsed from parameters (last resort; it's a runtime override, not the model max). Capabilities: vision/images → image input, audio → audio input, thinking → reasoning: true, tools/function_calling/tool_calling → toolCall: true. Unknown ids / malformed bodies → {} no-op.
  • src/core/enricher-resolver.ts — DEFAULT_OLLAMA_ENDPOINT = '/api/show', new modelIds option, Ollama branch fetches POST {origin}/api/show for every discovered id in parallel (per-request AbortSignal.timeout, default 5s unchanged). Per-model failures (404/timeout/malformed) are logged and degrade to no enrichment for that model; all failed → no enricher; never crashes discovery.
  • src/plugin/enhance-config.ts (V1) — the models list is already fetched before the resolver runs, so the diff is a single line: pass modelIds.
  • src/v2/discovery.ts (V2) — model fetch is sequentialized only when modelInfoFormat is Ollama (IDs are needed before /api/show can be called); payload validation extracted into a shared parseModelsPayload helper; all other formats keep the existing parallel Promise.all path untouched.

Explicit opt-in, no auto-detection

Consistent with every other format: users enable enrichment with modelInfoFormat: "ollama"; nothing is probed unless configured. Endpoint override via modelInfoEndpoint (absolute URL or origin-relative path).

{
  // V1
  "provider": {
    "ollama-local": {
      "options": {
        "baseURL": "http://localhost:11434/v1",
        "modelsDiscovery": { "enabled": true },
        "modelInfoFormat": "ollama"
      }
    }
  }
  // V2: per-provider discovery options { endpoint, modelInfoFormat, modelInfoInfoEndpoint }
}

Tests

26 new network-free tests (mocked fetch), test/fixtures/ollama/show-gemma.json is a trimmed live /api/show fixture:

  • enricher unit tests (test/ollama-model-info.test.ts): family-keyed model_info, family-preference over distractor families, top-level fallback, num_ctx variants (num_ctx 4096, num_ctx: 8192, "num_ctx": 16384), no-limit no-op, capability mapping, garbage tolerance, plus a V1 pipeline thread-through test.
  • resolver tests (test/core/enricher-resolver.test.ts): POST shape + origin URL resolution, per-model 404 tolerance, all-fail → undefined, missing/empty modelIds → guarded with 0 fetches, rejection mix, default vs custom endpoint.
  • V2 integration tests (test/v2/enrichment.test.ts): sequential fetch + per-model enrichment end-to-end, and all-/api/show-fail degradation with the V2 200k fallback intact.

Verification

  • npm run test:run — 41 files / 336 tests, all green (baseline 40/310).
  • npx tsc --noEmit — clean.
  • eslint src — 0 errors (20 pre-existing no-console warnings unchanged).
  • Verified against a live Ollama server (0.40.4-xollama.4): the V1 flow discovered 5/5 chat models with context lengths matching raw POST /api/show values exactly (131072 / 131072 / 524288 / 262144 / 131072), capabilities mapped correctly (vision/audio → modalities, thinking → reasoning, tools → tool-call), and the embedding model correctly excluded.

Fixes #162

Amendment (2026-10-19): num_ctx now wins over model_info context_length

After review against live Ollama data, the context-length resolution order in resolveOllamaContextLength changed: a valid num_ctx parsed from show.parameters is now the first source, above model_info.<details.family>.context_length / any *.context_length / top-level context_length, which remain the fallback chain and also apply when num_ctx is missing or non-positive.

Rationale: a num_ctx line in the model's Modelfile is the only intentional effective-context signal Ollama exposes; the model_info values are architecture-derived maxima. When a model is explicitly pinned (e.g. ling3-tiny-64k-prefill with num_ctx 64000 over an architecture max of 131072), the effective context is the pinned value. Models without num_ctx (e.g. Swift-Bonsai-2, gemma-4) are unaffected and keep their architecture maximum. Known accepted tradeoff: embedding models such as nomic-embed-text commonly pin num_ctx 8192 above their 2048 maximum and now report 8192; num_ctx is not clamped to the architecture max. (In practice such models are filtered from chat discovery by embedding classification, so the tradeoff rarely surfaces.)

New tests pin both directions: num_ctx wins over all context_length sources; a non-positive num_ctx falls back to the context_length chain. Docs (docs/configuration.md, docs/providers.md) updated to the new order.

Suite at HEAD of this PR: 338/338 tests across 41 files (tsc clean, lint 0 errors);
the "336" figure above reflects the state before the amendment commits.

…ember

Adds the Ollama enum member, the data-style sync enricher for POST /api/show bodies (family-keyed model_info context length, top-level fallback, num_ctx last resort, capability mapping), and registry wiring.

Deviation from plan §6 Step 3: test/ollama-model-info.test.ts is committed whole, so it also carries the V1 pipeline test block that logically belongs to the later V1 commit (the V1 source change lands separately).
…per-model failure tolerance

Adds DEFAULT_OLLAMA_ENDPOINT ('/api/show'), the optional modelIds resolution option, and the Ollama branch: parallel per-model POSTs, per-model failure tolerance (404/malformed/reject/timeout degrade to no enrichment for that model), Bearer auth when an apiKey is present, and per-request AbortSignal.timeout. Empty or omitted modelIds warns and returns undefined.
Passes modelIds (filtered through isDiscoveredRawModel) in the resolveModelInfoEnricher options so the Ollama format can POST /api/show per discovered model. Cached/persisted-model branches are untouched.

Deviation from plan §6 Step 6: src-only commit; the V1 pipeline test rode along in commit 1 (test/ollama-model-info.test.ts was committed whole).
…lama format

Extracts the duplicated payload parsing into parseModelsPayload and adds an Ollama branch that fetches /v1/models first, then resolves the enricher with the discovered model ids. All non-Ollama formats keep the existing parallel Promise.all path; the cached path is untouched.
Adds the ollama format to README's supported list, a configuration.md table row plus an Ollama Model Show subsection, a providers.md Ollama section, and the matching discovery-command prompt bullets in commands.ts.
@yuhp

yuhp commented Oct 19, 2026

Copy link
Copy Markdown
Owner

Thank you @DjFabFab for the detailed work and thoughtful contribution! Supporting Ollama's /api/show for accurate context lengths and capabilities is a very valuable and frequently requested feature.

We reviewed the implementation and would like to suggest a design adjustment regarding module cohesion:

Currently, to accommodate Ollama's per-model detail queries, the PR introduces format-specific branching directly into the core discovery engine (src/v2/discovery.ts with else if (config.modelInfoFormat === ModelInfoFormat.Ollama)), breaking the uniform concurrent scheduling of model discovery and tightly coupling the core pipeline to Ollama's specific querying lifecycle.

We'd love to see this implemented in a more cohesive, decoupled manner (高内聚):

  1. Keep the core discovery pipeline format-agnostic: Avoid adding format-specific if-else branches in src/v2/discovery.ts or src/plugin/enhance-config.ts.
  2. Encapsulate format-specific resolution: Keep the multi-step or per-model fetching logic encapsulated within the enricher/resolver domain (or design a generic deferred/batched enrichment contract that any provider can benefit from without hardcoding specific formats in the core engine).

Since we are currently freezing and finalizing the v2.0.0 stable release, we want to keep the core discovery loop minimal and stable. We would be very glad to iterate with you on this decoupled approach and target it for the upcoming v2.1.0 release!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

No metadata enrichment path for Ollama — context lengths and capabilities never discovered

2 participants