Skip to content

feat(provider): let custom providers set sampling and token knobs - #1308

Merged
TheGreatAxios merged 3 commits into
mainfrom
cl-9834-let-custom-openai-compatible-providers-set-context-window
Oct 3, 2026
Merged

TheGreatAxios merged 3 commits into
mainfrom
cl-9834-let-custom-openai-compatible-providers-set-context-window

Conversation

@TheGreatAxios

@TheGreatAxios TheGreatAxios commented Oct 3, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

  • Custom OpenAI-compatible providers configure context window, max output tokens, temperature, and top p; blank fields keep existing defaults and temperature zero remains valid.
  • Saved settings feed request defaults and refresh live context-window/compaction limits, including model switching, discovery, edits, removal, and duplicate-model precedence.
  • Token counts require positive integers; sampling ranges and mutually exclusive temperature/top-p are validated before connection or persistence. Preset and OAuth paths ignore hidden custom fields.

Verification

  • bun run check passes against current main: 8,254 tests, zero failures.
  • Failing-then-passing regressions cover invalid settings, persistence, hidden-field isolation, live context limits, worker/candidate isolation, and model switching during delayed discovery.
  • Independent review findings are resolved; no live provider API inference runs.

Fixes CL-9834

@linear-code

linear-code Bot commented Oct 3, 2026

Copy link
Copy Markdown

CL-9834

@TheGreatAxios TheGreatAxios left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review · Comment

Custom OpenAI-compatible providers persist optional context window, max output tokens, temperature, and top-p; the form rejects setting both sampling knobs.

Findings

  • src/config/index.ts:280 — buildOpenAISource writes both temperature and top_p into providerOptions when both fields are set. Mutual exclusion exists only in src/tui/provider/submit.ts:84. Hand-edited settings.json (or a catalog row) with both knobs reaches the outbound request. Verified: buildInferenceSourceForRef with temperature: 0.5 and topP: 0.8 yields { temperature: 0.5, top_p: 0.8 }. Refuse the pair in buildOpenAISource / the settings schema, not only at submit.

  • src/tui/provider/form.ts:69 — stepReady has no lower bound for maxTokens or contextWindow. "0" and "-1" are ready, submit persists them, and fields.maxTokens ?? SOURCE_MAX_TOKENS keeps 0 (it is not undefined). A zero/negative output cap starves generation; a zero/negative context window poisons compaction and the status-bar meter. Reject non-positive integers at stepReady and at submit, matching the 0–2 / 0–1 checks already used for temperature / top-p.

@TheGreatAxios TheGreatAxios left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review findings resolved at 9f16963: live context limits refresh without a restart, hidden custom numeric fields cannot affect preset/OAuth submissions, and invalid/noninteger token counts are rejected by shared ArkType validation. A further integration regression proves delayed discovery cannot undo a model switch. Test-first evidence: original regressions fail before fixes, focused checks pass afterward, and full bun run check against current main passes with 8,254 tests and zero failures. All original PR files and follow-up changes are reviewed; no remaining actionable findings. No live API smoke is claimed.

@TheGreatAxios
TheGreatAxios merged commit 25e246b into main Oct 3, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant