feat(models): openai-compatible vendor presets (zai, deepseek, moonshot, groq, xai, openrouter) by fjgbue · Pull Request #17 · extra-org/extra

fjgbue · 2026-07-14T15:30:04Z

The openai provider was hardcoded to api.openai.com. First pass just added base_url and api_key_env so provider: openai could point anywhere, but that means typing out the endpoint and key var by hand every time, even for vendors this table already knows about. This adds the actual integration on top of that: agent_engine/models/presets.py is a small registry mapping provider ids (zai, deepseek, moonshot, groq, xai, openrouter) to their base_url/api_key_env, resolved in build_chat_model before it dispatches to _build_openai_model. No new provider kind, no new dependency, it's ChatOpenAI either way.

model:
provider: zai
name: glm-5.2

That's the whole config now for a listed vendor. api_key_env still gets checked (ZAI_API_KEY in this case), it's just not typed into the YAML anymore. base_url/api_key_env still work directly, both as the escape hatch for anything not in the table (a different vendor, self-hosted Ollama/vLLM) and as an override on a listed preset, e.g. routing zai through an internal proxy without giving up the shorthand:

model:
provider: zai
name: glm-5.2
base_url: https://llm-proxy.internal.example.com/zai
api_key_env: INTERNAL_PROXY_KEY

Also fixed ARCHITECTURE.md and ROADMAP.md, which still only mentioned Anthropic and Bedrock even though Gemini and OpenAI have been in factory.py for a while, and updated both again now that there's an actual preset registry instead of just two generic fields.

Tests: 379 passing (up from 370), including a parametrized case that checks every preset resolves its own base_url/key correctly, one confirming an explicit base_url/api_key_env override wins over the preset, one for case-insensitive provider ids, and one confirming the unsupported-provider error message lists the preset ids alongside anthropic/bedrock/gemini/openai. ruff and mypy clean on every file touched.

…_key_env The openai provider only ever targeted api.openai.com. Add optional base_url and api_key_env fields to ModelConfig so the same provider covers any OpenAI-compatible vendor (Z.AI, DeepSeek, Moonshot, Groq, xAI, OpenRouter) or self-hosted server (Ollama, vLLM) without a new provider integration or dependency. api_key_env defaults to OPENAI_API_KEY, so existing YAML is unaffected.

YAML_SPEC.md now documents the new OpenAI-compatible endpoint fields with a table of known vendors. ARCHITECTURE.md and ROADMAP.md previously only mentioned Anthropic + Bedrock as model providers, even though Gemini and OpenAI have been supported in factory.py for a while — updated both to reflect what's actually implemented.

base_url/api_key_env alone meant typing out both fields by hand every time, even for vendors this codebase already knows about. Add agent_engine/models/presets.py, a small registry mapping provider ids (zai, deepseek, moonshot, groq, xai, openrouter) to their base_url and api_key_env, resolved in build_chat_model before dispatch. provider: zai now works with nothing else set; base_url/api_key_env still override a preset explicitly, or work standalone on provider: openai for any vendor or self-hosted server not in the table.

YAML_SPEC.md leads with provider: zai (etc) as the primary path now, keeps base_url/api_key_env as the fallback for anything not in the table plus the override mechanism. ARCHITECTURE.md and ROADMAP.md updated to match, they still described the plain generic fields from the previous commit.

AmitAvital1 · 2026-07-16T16:25:31Z

+    # a local Ollama/vLLM server, ...) by naming the env var that holds that
+    # vendor's key. Defaults to OPENAI_API_KEY to keep existing YAML working
+    # unchanged.
+    key_env_var = api_key_env.strip() if api_key_env and api_key_env.strip() else "OPENAI_API_KEY"


If api_key_env is "" we not should error so user meant to override and not use OPEN_API_KEY?

empty string should error, yeah.

the OPENAI_API_KEY default is meant for when the field is omitted, not when someone explicitly blanks it. An explicit "" means they were mid-override and got it wrong, and falling back silently just turns that into a confusing auth failure further down. I'll split the two: omitted stays defaulted, empty or whitespace raises. what u think?

I still wondering if user will declare on yaml api_key_env = " "
we will fall back to open api key default. but, he tried to override and maybe typo, so user experience in my view should error him.
Now we fallback

AmitAvital1 · 2026-07-16T16:36:16Z

@@ -0,0 +1,70 @@
+"""Known OpenAI-compatible vendor presets.


I'm not sure this file is needed. what you have built, its generic infra for custom models.
This led us to maintain code and each changes of the providers. let user declare api key name and urls and models, they write once time yaml and use model name each place

the maintenance worry is fair but smaller than it looks I think

these base URLs don't move, and the table is six entries. The generic base_url/api_key_env path already covers everything the registry doesn't, including overriding a preset, so the registry adds no capability, it just saves every user re-typing the same two known values for a vendor the library could already name, or at least that was my initial tought. But if the team would rather ship zero vendor knowledge though, killing presets.py is one commit since the generic path stands on its own, lmk

@Asaf-prog wdyt? we need more opinion

Secondly i feel good with that. But we have to doc it also on official docs mdx files. focused and clear

AmitAvital1 · 2026-07-16T16:47:54Z

+    if preset is not None:
+        base_url = base_url or preset.base_url
+        api_key_env = api_key_env or preset.api_key_env
+        normalized_provider = "openai"


if we remove preset always provider will be openai

yes, right, drop the registry and this branch goes with it, proider is just openai plus whatever base_url/key the user passed. they live or die together

AmitAvital1 · 2026-07-16T16:48:42Z

    if not model_name:
        raise ModelConfigurationError("Model name must not be empty.")

+    # A preset id (zai, deepseek, moonshot, groq, xai, openrouter, ...) is


what about checking if provider is gemini, but user declare base url or api key, or just one of them? we should error him on validate yaml that its not acceptable

fjgbue

Awaiting some further comments in the answers!

fjgbue · 2026-07-16T20:15:34Z

@@ -0,0 +1,70 @@
+"""Known OpenAI-compatible vendor presets.


the maintenance worry is fair but smaller than it looks I think

these base URLs don't move, and the table is six entries. The generic base_url/api_key_env path already covers everything the registry doesn't, including overriding a preset, so the registry adds no capability, it just saves every user re-typing the same two known values for a vendor the library could already name, or at least that was my initial tought. But if the team would rather ship zero vendor knowledge though, killing presets.py is one commit since the generic path stands on its own, lmk

fjgbue · 2026-07-16T20:29:45Z

+    if preset is not None:
+        base_url = base_url or preset.base_url
+        api_key_env = api_key_env or preset.api_key_env
+        normalized_provider = "openai"


yes, right, drop the registry and this branch goes with it, proider is just openai plus whatever base_url/key the user passed. they live or die together

anthropic, bedrock and gemini target a fixed endpoint and their own key, so base_url/api_key_env passed alongside them were silently dropped in the factory. _validate_model now errors at yaml-load time, naming the field and provider, with each field checked independently so setting just one is caught too.

The yaml secret scanner flags any key whose name looks secret-like, and api_key_env contains "api_key", so it tripped the same check max_tokens was already exempt from. That left the base_url/api_key_env escape hatch unusable from yaml. Exempt the key name; values are still scanned, so a real key pasted there is still rejected.

AmitAvital1 · 2026-07-17T09:53:59Z

+                    ValidationError(
+                        f"{path}.{field_name}",
+                        f"Not supported for provider '{provider}'; only OpenAI-compatible "
+                        "providers accept a custom endpoint or key variable.",


on error message: base url or api key naming variable?

AmitAvital1 · 2026-07-17T09:57:44Z

            )
        )

+    if provider in _FIXED_ENDPOINT_PROVIDERS:


instead of nested ifs lets try make it better like (pseudo) :
for field_name in ("base_url", "api_key_env"):
if raw.get(field_name) is not None && provider in _FIXED_ENDPOINT_PROVIDERS
return error

AmitAvital1 · 2026-07-18T07:28:57Z

https://docs.extra-ai.co/docs/yaml-spec#supported-providers

We should documented this new feature.
Now we are doing supported providers on yaml refernce. i think we should add new section under features calling providers. detailed the pre defined providers, and supported open ai api providers and custom providers and how defining them

and here just keep example but add reference to this new section https://docs.extra-ai.co/docs/yaml-spec#supported-providers

AmitAvital1 · 2026-07-20T19:09:53Z

Hi, want share with you so i checked and our engine are supporting by that using langchain open api support by using this env var

like ollama
OPENAI_API_KEY=ollama
OPENAI_BASE_URL=http://host.docker.internal:11434/v1

open router
OPENAI_API_KEY=sk-*****
OPENAI_BASE_URL=https://openrouter.ai/api/v1

we just put provider in
defaults:
model:
provider: openai
name: nvidia/llama-nemotron-rerank-vl-1b-v2:free
temperature: 0.0

so lets think how its affected since, we currently support it. but need to check how its works, and if we decide go with your fix make sure we have no collisions

fjgbue added 2 commits July 14, 2026 17:29

fjgbue marked this pull request as draft July 14, 2026 15:41

fjgbue added 2 commits July 14, 2026 17:51

fjgbue changed the title ~~feat(models): support any OpenAI-compatible endpoint via base_url/api_key_env~~ feat(models): openai-compatible vendor presets (zai, deepseek, moonshot, groq, xai, openrouter) Jul 14, 2026

Asaf-prog requested review from AmitAvital1 and Asaf-prog July 14, 2026 16:02

Asaf-prog assigned Asaf-prog and fjgbue and unassigned Asaf-prog Jul 14, 2026

fjgbue marked this pull request as ready for review July 14, 2026 16:06

AmitAvital1 reviewed Jul 16, 2026

View reviewed changes

Comment thread src/agent_engine/models/factory.py

AmitAvital1 reviewed Jul 16, 2026

View reviewed changes

AmitAvital1 requested changes Jul 16, 2026

View reviewed changes

fjgbue commented Jul 16, 2026

View reviewed changes

fjgbue marked this pull request as draft July 16, 2026 20:45

fjgbue added 2 commits July 16, 2026 22:52

fjgbue force-pushed the feat/openai-compat-providers branch from d64c1e5 to 0a9d92e Compare July 16, 2026 20:54

AmitAvital1 reviewed Jul 17, 2026

View reviewed changes

Uh oh!

Conversation

fjgbue commented Jul 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

fjgbue left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

AmitAvital1 Jul 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

AmitAvital1 commented Jul 18, 2026

Uh oh!

AmitAvital1 commented Jul 20, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

fjgbue commented Jul 14, 2026 •

edited

Loading

AmitAvital1 Jul 17, 2026 •

edited

Loading