You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Autocomplete silently produces nothing when Ollama Modelfile num_ctx equals Continue's default maxTokens (4096) — pruneLength computes a negative prompt budget and prunes prefix/suffix to empty #13038
GPU: NVIDIA MX250 (2 GB VRAM) — relevant only because it motivated me to pin num_ctx to 4096 to keep the model fully in VRAM
config (~/.continue/config.yaml):
name: Local Agentversion: 1.0.0schema: v1models:
- name: Qwen 2.5 Coder 1.5b AllGPUprovider: ollamamodel: qwen2.5_coder-1.5b:allgpuroles:
- autocomplete
- chat# NOTE: no explicit contextLength / maxTokens here — both fall back to defaults
- name: Autodetectprovider: ollamamodel: AUTODETECTroles:
- autocomplete
- chat
- edit
- applycontext:
- provider: code
- provider: docs
- provider: diff
- provider: terminal
- provider: problems
- provider: folder
- provider: codebase
The Ollama Modelfile for qwen2.5_coder-1.5b:allgpu contains:
FROM <…qwen2.5-coder-1.5b-instruct-q4_k_m.gguf…>
PARAMETER num_ctx 4096
PARAMETER num_gpu 32
For comparison, the qwen2.5_coder-1.5b:latest tag has nonum_ctx line in its Modelfile (everything else identical, same weight blob).
Description
Tab autocomplete silently produces nothing — no completion item, no error message, no log line — when the Ollama Modelfile for the selected autocomplete model contains PARAMETER num_ctx 4096 and the user has not explicitly overridden maxTokens in config.yaml. The same model tag without the num_ctx line (:latest) autocomplete works fine.
Direct curl to http://localhost:11434/api/generate with identical prompts returns correct completions for both tags, so the Ollama side is healthy. The failure is 100% client-side, inside Continue's autocomplete pipeline.
Root cause (traced in the bundled extension.js)
When the Ollama provider is instantiated, Continue calls POST /api/show and parses the parameters string line by line (extension.js:731409–731451). A num_ctx line overwrites this._contextLength:
If the user did not set contextLength in config.yaml, the Modelfile value wins. With num_ctx 4096 in the Modelfile, Continue sees contextLength = 4096.
If the user did not set maxTokens either, maxTokens falls back to DEFAULT_MAX_TOKENS = 4096 (extension.js:239811).
For models without a num_ctx line in their Modelfile, contextLength falls back to DEFAULT_CONTEXT_LENGTH = 32768 (extension.js:239813 + the getter at 243810).
The autocomplete pipeline then calls pruneLength (extension.js:252546):
There is no Math.max(0, maxAllowedPromptTokens) clamp. Substituting the two tags:
Tag
contextLength
maxTokens
safetyBuffer
maxAllowedPromptTokens
prune for any non-empty prompt
:latest (no num_ctx)
32768
4096
655.36
28017
negative → no pruning ✅
:allgpu (num_ctx 4096)
4096
4096
81.92
−81.92
always positive → always over-prunes ❌
renderPromptWithTokenLimit (extension.js:252585–252624) enters the prune > 0 branch and computes:
constallowedPrefixTokens=Math.max(0,prefixTokenCount-dropPrefix);// = 0 when dropPrefix > prefixTokenCountconstallowedSuffixTokens=Math.max(0,suffixTokenCount-dropSuffix);// = 0 when dropSuffix > suffixTokenCountprefix=pruneLinesFromTop(prefix,allowedPrefixTokens,helper.modelName);suffix=pruneLinesFromBottom(suffix,allowedSuffixTokens,helper.modelName);
pruneLinesFromTop(prompt, 0, …) (extension.js:243283–243297) loops while (totalTokens > 0 && start < length) start++ until start === length, then returns lines.slice(length).join("\n") — the empty string. Same for suffix.
The request is then sent to Ollama with prefix === "" and suffix === "\n" (the fallback). The model has no anchor and emits an empty or garbage completion. postprocessCompletion (extension.js:251045) drops it via if (isBlank(completion)) return void 0;, and provideInlineCompletionItems returns undefined (extension.js:254898). No error, no log, no UI feedback.
Note that _streamFim (extension.js:731688–731722) does not gate on contextLength and always fires the request, and num_ctx: this.contextLength (extension.js:731507) is what Continue POSTs back to Ollama as the KV cache size — so the bug is purely in the client-side prompt-budget arithmetic, not in the Ollama round-trip.
#7544 is the same family (contextLength − maxTokens − safetyBuffer goes negative or too small) but in the edit pipeline, where it surfaces as a visible Error streaming edit diffs: Token limit reached. File/range likely too large for this edit. This issue is the autocomplete variant, which has no such error path — the failure is completely silent. The config validator at extension.js:239188 only checks model.defaultCompletionOptions fields that the user wrote explicitly; it does not validate the _contextLength value derived from /api/show, so no warning is emitted either.
#7702 (merged in Continue 1.2.11) overrides the Ollama contextLength getter to fall back to 8192 instead of 32768 when no num_ctx is present. That helps the no-num_ctx case only when the user has not set contextLength in config. It does not help any Modelfile that explicitly sets num_ctx 4096 (Continue still reads 4096 via /api/show), and it does not touch the pruneLength arithmetic at extension.js:252550. So upgrading Continue does not fix this case.
To reproduce
Pull a small coder model in Ollama and create two tags — one with num_ctx 4096 pinned, one without:
When contextLength is less than or equal to reservedTokens + safetyBuffer, Continue should not compute a negative prompt budget and silently prune the prompt to empty. At minimum:
pruneLength should clamp maxAllowedPromptTokens to Math.max(0, …) (or MIN_RESPONSE_TOKENS), so the worst case is "badly trimmed but still sends something", not "sends nothing and tells no-one".
The config validator at extension.js:239188 should also validate the /api/show-derived _contextLength vs maxTokens and emit a non-fatal warning so users at least see that their inputs are being truncated.
postprocessCompletion should distinguish "blank because the model produced nothing useful" from "blank because the client sent an empty prompt"; the latter should log a warning.
Workarounds (for anyone hitting this today)
Any one of these fixes it:
In config.yaml, give the model an explicit small maxTokens so the budget is positive:
In config.yaml, give the model an explicit larger contextLength (Continue will POST it back to Ollama as num_ctx, so the server KV cache must actually fit):
defaultCompletionOptions:
contextLength: 8192
Remove the PARAMETER num_ctx 4096 line from the Ollama Modelfile (Continue will fall back to DEFAULT_CONTEXT_LENGTH).
No log output is produced by this bug — that is part of the problem. The Continue output channel shows normal stream providerID=ollama modelID=… step=… lines and then a normal done for Ollama's response; the truncation happens before the request and the discard happens after, both silently.
Hello, I looked at the autocomplete silence you get when Ollama num_ctx is 4096 and Continue falls back to the default maxTokens of 4096.
pruneLength going negative is expected in that case: Continue thinks there is no room left for the prefix and suffix, so it prunes both to empty and the model returns nothing, with no error. A practical fix is to set contextLength or maxTokens on that model in ~/.continue/config.yaml to a value clearly below the Ollama num_ctx (or raise num_ctx if VRAM allows), so the prompt budget stays positive. Your Qwen 2.5 Coder 1.5b AllGPU entry currently leaves both unset, which is why it hits the default 4096 against the pinned num_ctx.
This is a budget bug, not a missing model, so I would not change the Ollama Modelfile alone without also setting those Continue fields.
Activity
japric commentedon Jul 27, 2026
Before submitting your bug report
pruneLength,num_ctx,autocomplete silent,contextLength maxTokens; closest hit is Error streaming edit diffs: Token limit reached. File/range likely too large for this edit #7544 which is the edit-mode variant with a visible error — this one is the autocomplete-mode variant with no error and no output at all)Relevant environment info
qwen2.5-coder:1.5b(Ollama 0.13.5, local)num_ctxto 4096 to keep the model fully in VRAM~/.continue/config.yaml):The Ollama Modelfile for
qwen2.5_coder-1.5b:allgpucontains:For comparison, the
qwen2.5_coder-1.5b:latesttag has nonum_ctxline in its Modelfile (everything else identical, same weight blob).Description
Tab autocomplete silently produces nothing — no completion item, no error message, no log line — when the Ollama Modelfile for the selected autocomplete model contains
PARAMETER num_ctx 4096and the user has not explicitly overriddenmaxTokensinconfig.yaml. The same model tag without thenum_ctxline (:latest) autocomplete works fine.Direct
curltohttp://localhost:11434/api/generatewith identical prompts returns correct completions for both tags, so the Ollama side is healthy. The failure is 100% client-side, inside Continue's autocomplete pipeline.Root cause (traced in the bundled
extension.js)When the Ollama provider is instantiated, Continue calls
POST /api/showand parses theparametersstring line by line (extension.js:731409–731451). Anum_ctxline overwritesthis._contextLength:If the user did not set
contextLengthinconfig.yaml, the Modelfile value wins. Withnum_ctx 4096in the Modelfile, Continue seescontextLength = 4096.If the user did not set
maxTokenseither,maxTokensfalls back toDEFAULT_MAX_TOKENS = 4096(extension.js:239811).For models without a
num_ctxline in their Modelfile,contextLengthfalls back toDEFAULT_CONTEXT_LENGTH = 32768(extension.js:239813+ the getter at243810).The autocomplete pipeline then calls
pruneLength(extension.js:252546):There is no
Math.max(0, maxAllowedPromptTokens)clamp. Substituting the two tags:contextLengthmaxTokenssafetyBuffermaxAllowedPromptTokensprunefor any non-empty prompt:latest(nonum_ctx):allgpu(num_ctx 4096)renderPromptWithTokenLimit(extension.js:252585–252624) enters theprune > 0branch and computes:pruneLinesFromTop(prompt, 0, …)(extension.js:243283–243297) loopswhile (totalTokens > 0 && start < length) start++untilstart === length, then returnslines.slice(length).join("\n")— the empty string. Same for suffix.The request is then sent to Ollama with
prefix === ""andsuffix === "\n"(the fallback). The model has no anchor and emits an empty or garbage completion.postprocessCompletion(extension.js:251045) drops it viaif (isBlank(completion)) return void 0;, andprovideInlineCompletionItemsreturnsundefined(extension.js:254898). No error, no log, no UI feedback.Note that
_streamFim(extension.js:731688–731722) does not gate oncontextLengthand always fires the request, andnum_ctx: this.contextLength(extension.js:731507) is what Continue POSTs back to Ollama as the KV cache size — so the bug is purely in the client-side prompt-budget arithmetic, not in the Ollama round-trip.Why this is distinct from #7544
#7544 is the same family (
contextLength − maxTokens − safetyBuffergoes negative or too small) but in the edit pipeline, where it surfaces as a visibleError streaming edit diffs: Token limit reached. File/range likely too large for this edit. This issue is the autocomplete variant, which has no such error path — the failure is completely silent. The config validator atextension.js:239188only checksmodel.defaultCompletionOptionsfields that the user wrote explicitly; it does not validate the_contextLengthvalue derived from/api/show, so no warning is emitted either.Why #7702 does not fix this
#7702 (merged in Continue 1.2.11) overrides the Ollama
contextLengthgetter to fall back to8192instead of32768when nonum_ctxis present. That helps the no-num_ctxcase only when the user has not setcontextLengthin config. It does not help any Modelfile that explicitly setsnum_ctx 4096(Continue still reads 4096 via/api/show), and it does not touch thepruneLengtharithmetic atextension.js:252550. So upgrading Continue does not fix this case.To reproduce
Pull a small coder model in Ollama and create two tags — one with
num_ctx 4096pinned, one without:In
~/.continue/config.yamladd one model entry withroles: [autocomplete]and no explicitcontextLengthormaxTokens:Reload VSCode (
Developer: Reload Window).Open a small Python file, e.g.:
Trigger autocomplete at the cursor. No completion appears. No error in the Continue output channel.
Switch the autocomplete model to
qwen2.5-coder:1.5b(the:latesttag withoutnum_ctx).Reload VSCode. Trigger autocomplete at the same cursor.
hello()is suggested.Direct API sanity check (both tags return the same correct completion):
Expected behavior
When
contextLengthis less than or equal toreservedTokens + safetyBuffer, Continue should not compute a negative prompt budget and silently prune the prompt to empty. At minimum:pruneLengthshould clampmaxAllowedPromptTokenstoMath.max(0, …)(orMIN_RESPONSE_TOKENS), so the worst case is "badly trimmed but still sends something", not "sends nothing and tells no-one".extension.js:239188should also validate the/api/show-derived_contextLengthvsmaxTokensand emit a non-fatal warning so users at least see that their inputs are being truncated.postprocessCompletionshould distinguish "blank because the model produced nothing useful" from "blank because the client sent an empty prompt"; the latter should log a warning.Workarounds (for anyone hitting this today)
Any one of these fixes it:
In
config.yaml, give the model an explicit smallmaxTokensso the budget is positive:In
config.yaml, give the model an explicit largercontextLength(Continue will POST it back to Ollama asnum_ctx, so the server KV cache must actually fit):Remove the
PARAMETER num_ctx 4096line from the Ollama Modelfile (Continue will fall back toDEFAULT_CONTEXT_LENGTH).Related issues / PRs
num_ctxdefault but does not touchpruneLengthcontextLengthvsnum_ctxinconsistency between Continue and Ollamaparameters-string parser (stop tokens), showing this code path has historynum_ctx/num_gpuoptions not being respectedLog output
No log output is produced by this bug — that is part of the problem. The Continue output channel shows normal
stream providerID=ollama modelID=… step=…lines and then a normaldonefor Ollama's response; the truncation happens before the request and the discard happens after, both silently.fix: clamp maxAllowedPromptTokens to zero in autocomplete pruner
fix: clamp maxAllowedPromptTokens to zero in autocomplete pruner
fix(autocomplete): cap reserved output tokens in pruneLength
corbenicai commentedon Nov 16, 2026
Hello, I looked at the autocomplete silence you get when Ollama
num_ctxis 4096 and Continue falls back to the defaultmaxTokensof 4096.pruneLengthgoing negative is expected in that case: Continue thinks there is no room left for the prefix and suffix, so it prunes both to empty and the model returns nothing, with no error. A practical fix is to setcontextLengthormaxTokenson that model in~/.continue/config.yamlto a value clearly below the Ollamanum_ctx(or raisenum_ctxif VRAM allows), so the prompt budget stays positive. Your Qwen 2.5 Coder 1.5b AllGPU entry currently leaves both unset, which is why it hits the default 4096 against the pinnednum_ctx.This is a budget bug, not a missing model, so I would not change the Ollama Modelfile alone without also setting those Continue fields.