feat: add OrcaRouter as a named OpenAI-compatible provider - #1955
XiaoHuo888-hue wants to merge 1 commit into
Conversation
Add a first-class OrcaRouter provider mirroring the MiniMax wiring: - OrcaRouterModels constants (orcarouter/auto, fusion, fusion-flash, fusion-mini) - embabel-agent-orcarouter-autoconfigure module registering four LlmService beans via OpenAiCompatibleModelFactory, base URL https://api.orcarouter.ai/v1 - embabel-agent-starter-orcarouter starter - OpenAiCompatibleModelFactory.orcaRouter(apiKey) BYOK spec + endpointFor support - BYOK credential endpoint coverage, README env vars, docs (installing + reference) Co-Authored-By: jinhao.song <jinhao.song@myflashcloud.com> Signed-off-by: XiaoHuo888-hue <jinhao.song@myflashcloud.com>
|
@XiaoHuo888-hue - thank you very much for contributing! Please advise on priority. |
jasperblues
left a comment
jasperblues
left a comment
There was a problem hiding this comment.
Requesting changes on two points. Build and tests are green locally (embabel-agent-openai, embabel-agent-byok-autoconfigure and the new orcarouter module, 79 tests, no failures), and the wiring is thorough: every parametrized provider test in the repo has been found and updated, and the doc page is proportionate to the minimax and zai pages.
Posting this ahead of the issue, on the assumption it gets approved and allocated to a milestone. For future contributions, please note @igordayen's earlier point that this project wants an issue opened before a PR is raised.
| https://www.orcarouter.ai[OrcaRouter] is a model gateway that routes to models from multiple vendors through one OpenAI-compatible API. | ||
| Embabel integrates OrcaRouter as a first-class provider using the same `OpenAiCompatibleModelFactory` pattern as other OpenAI-compatible providers (MiniMax, Atlas Cloud). | ||
|
|
||
| It also runs gateway-level, zero-trust security for AI agents on the same endpoint — screening every prompt/response and governing every tool call on a default-deny basis, with no application code changes. |
There was a problem hiding this comment.
This states a security guarantee that Embabel cannot verify or test, and placing it in reference/ publishes it as an Embabel statement about how the framework behaves rather than as a vendor claim.
It also sits alongside reference/guardrails, which is Embabel's own mechanism. A reader can reasonably come away believing that prompt screening and default-deny tool governance are a supported security boundary of this framework, when they are a property of a third-party service on the other side of an HTTP call.
Please remove it. A neutral link to your own documentation carries the same information without the project vouching for it. This one is blocking.
| pricingModel = PricingModel.ALL_YOU_CAN_EAT, | ||
| retryTemplate = properties.retryTemplate(OrcaRouterModels.AUTO), | ||
| ) | ||
| } | ||
|
|
||
| @Bean | ||
| fun orcaRouterFusion(): LlmService<*> { | ||
| return openAiCompatibleLlm( | ||
| model = OrcaRouterModels.FUSION, | ||
| provider = OrcaRouterModels.PROVIDER, | ||
| knowledgeCutoffDate = null, | ||
| optionsConverter = OrcaRouterOptionsConverter, | ||
| pricingModel = PricingModel.ALL_YOU_CAN_EAT, | ||
| retryTemplate = properties.retryTemplate(OrcaRouterModels.FUSION), | ||
| ) | ||
| } | ||
|
|
||
| @Bean | ||
| fun orcaRouterFusionFlash(): LlmService<*> { | ||
| return openAiCompatibleLlm( | ||
| model = OrcaRouterModels.FUSION_FLASH, | ||
| provider = OrcaRouterModels.PROVIDER, | ||
| knowledgeCutoffDate = null, | ||
| optionsConverter = OrcaRouterOptionsConverter, | ||
| pricingModel = PricingModel.ALL_YOU_CAN_EAT, | ||
| retryTemplate = properties.retryTemplate(OrcaRouterModels.FUSION_FLASH), | ||
| ) | ||
| } | ||
|
|
||
| @Bean | ||
| fun orcaRouterFusionMini(): LlmService<*> { | ||
| return openAiCompatibleLlm( | ||
| model = OrcaRouterModels.FUSION_MINI, | ||
| provider = OrcaRouterModels.PROVIDER, | ||
| knowledgeCutoffDate = null, | ||
| optionsConverter = OrcaRouterOptionsConverter, | ||
| pricingModel = PricingModel.ALL_YOU_CAN_EAT, |
There was a problem hiding this comment.
All four beans are priced at zero: PricingModel.ALL_YOU_CAN_EAT is PerTokenPricingModel(0.0, 0.0) (PricingModel.kt:46).
This is the platform-key path rather than BYOK. The key comes from the deployment's ORCAROUTER_API_KEY, so the call is billed to the deployment. Zero is correct on the BYOK side, and the BYOK half of this PR is right as written: see the rationale at CredentialEndpoint.kt:66. Here it means reference/cost-tracking reports $0 for every OrcaRouter call, and anything reading those figures to make a budget decision sees a free model.
MiniMaxModelsConfig, which this file mirrors, sets real per-token prices. Z.ai and DashScope use perModelPrice ?: ALL_YOU_CAN_EAT, so zero is their fallback for an unrecognised model rather than a flat declaration for the provider.
A router does have genuinely variable per-request cost, so if published numbers do not exist for these four IDs, an explicit comment saying so would be fine. What is worth avoiding is the zero being inherited silently from the file this was copied from.
|
Following up on the issue creation for this PR. Could you please update? Thank you @jasperblues -FYI |
@XiaoHuo888-hue - how are you. following up... thanks |
3 participants
Summary
Add OrcaRouter as a named, first-class OpenAI-compatible provider, mirroring the existing MiniMax wiring.
OrcaRouter is a model gateway that routes to models from multiple vendors through one OpenAI-compatible endpoint (
https://api.orcarouter.ai/v1). It also runs gateway-level, zero-trust security for AI agents on the same endpoint — screening every prompt/response and governing every tool call on a default-deny basis, with no application code changes.This makes OrcaRouter selectable the same way MiniMax / Z.ai / Atlas Cloud are today: add the starter, set
ORCAROUTER_API_KEY, and the four gateway models (orcarouter/auto,orcarouter/fusion,orcarouter/fusion-flash,orcarouter/fusion-mini) appear asLlmServicebeans.Changes
embabel-agent-orcarouter-autoconfigure—OrcaRouterModelsConfigregisters fourLlmServicebeans viaOpenAiCompatibleModelFactory(base URLhttps://api.orcarouter.ai/v1, env varsORCAROUTER_API_KEY/ORCAROUTER_BASE_URL), plusOrcaRouterOptionsConverterand unit tests.embabel-agent-starter-orcarouter.OrcaRouterModels(orcarouter/auto,fusion,fusion-flash,fusion-mini).OpenAiCompatibleModelFactory.orcaRouter(apiKey)factory andendpointFor("OrcaRouter")support;CredentialEndpointConfigandProviderDetectiondocs updated.embabel-agent-autoconfigure,embabel-agent-starters,embabel-agent-dependencies.getting-started/installing,reference/orcarouter, modules starter table, testing env list.Validation
mvn testonembabel-agent-openai,embabel-agent-orcarouter-autoconfigure, andembabel-agent-byok-autoconfigure— all green (newOrcaRouterOptionsConverterTest8/8,AgentOrcaRouterAutoConfigurationTest1/1).OpenAiCompatibleModelFactory.orcaRouter(key).buildValidated()againsthttps://api.orcarouter.ai/v1with a real key returns a workingLlmService(HTTP 200, providerOrcaRouter).Disclosure: I'm an engineer on the OrcaRouter team.