[whyKirbyy]

Router submission: Corealis Router

Leaderboard name: Corealis Router. Affiliation: Corealis Consulting AG (https://corealis.ch). The file name corealis-router is only the prediction key.

Corealis Router (Corealis Consulting AG) is the router behind Corealis AI. A small router model (typesafe/jev-1.14) reads each request and picks one of three models: GLM-5.3, GLM-5.3 Flash or DeepSeek V4.1 Flash. It then picks how much the chosen model thinks. The @low / @high / @default suffix is that reasoning effort. This is the production version from 2026-10-22, run once on all 8,400 prompts through OpenRouter.

How it routes. The router is prompted, not trained. Its instructions are neutral ("pick the cheapest model that will answer it correctly"), and the model descriptions are written from size and price only. No RouterArena answers, scores or labels were used to fit the router.

Disclosure. Before freezing this version, we scored two earlier versions of our own router on a random quarter of RouterArena and on half of Arena-Hard v2.0. We chose this general version over the earlier one partly on those results. We also read other routers' published prediction files. Nothing was tuned per question or per dataset. Please let us know if you consider this out of bounds.

Files

• router_inference/config/corealis-router.json: the 5-entry pool (3 models × effort)
• router_inference/predictions/corealis-router.json: 8,400 rows with generated_result
• router_inference/predictions/corealis-router-robustness.json: 420 rows, picks only, from a real re-run on the perturbed prompts
• model_cost/model_cost.json: the 5 entries added at the cheapest OpenRouter list prices (USD per 1M tokens). Existing entries are untouched.
• universal_model_names.py: the same 5 names added

Local check. check_config_prediction_files.py passes for full and robustness. Local llm_evaluation/run.py: accuracy 74.85%, $0.79 per 1K queries, score 71.9, robustness 0.95. 78 prompts returned an empty answer (the chosen model hit its thinking limit). They are kept as failed generations (success: false).

Closed-source router. Contact: tim.schmid@corealis.ch

[whyKirbyy]

/evaluate

[github-actions]

Router Evaluation Results

Router: corealis-router
Dataset Split: full

RouterArena Metrics

Metric	Value
RouterArena Score	0.7191
Accuracy	74.85%
Total Cost	$6.637140
Avg Cost per Query	$0.000790
Avg Cost per 1K Queries	$0.7901
Number of Queries	8400
Abnormal Entries	78
Robustness Score	0.9500

⚠️ 78 of 8400 queries (0.9%) had no valid generation (inference failed / empty answer) and were scored as incorrect (0). These queries still count toward the denominator, so accuracy and cost reflect the full query set. Please regenerate predictions for these queries and resubmit for a complete evaluation.

---

Evaluation completed by RouterArena automated workflow

[yl231]

Thanks for the clear write-up, @whyKirbyy, and for disclosing how this version was chosen.

The README rule excludes tuning a router against RouterArena, and choosing between versions by their RouterArena score is a mild form of that. At the scale you describe (two versions, scored on a random quarter of the queries alongside Arena-Hard v2.0), we're fine with it as long as the choice doesn't depend on RouterArena. So one question:

Using Arena-Hard v2.0 alone, would you have picked the same version? If yes, we'll accept the submission as it is.

Separately, the scores above were computed on Oct 22, before three LiveCodeBench grading fixes landed on main (#239, #230, #234). We'll re-run /evaluate here with the current evaluator, so the numbers may shift slightly.

[yl231]

/evaluate

[github-actions]

Router Evaluation Results

Router: corealis-router
Dataset Split: full
Evaluator: main @ 9f6ae25

RouterArena Metrics

Metric	Value
RouterArena Score	0.7248
Accuracy	75.54%
Total Cost	$6.637140
Avg Cost per Query	$0.000790
Avg Cost per 1K Queries	$0.7901
Number of Queries	8400
Abnormal Entries	78
Robustness Score	0.9500

⚠️ 78 of 8400 queries (0.9%) had no valid generation (inference failed / empty answer) and were scored as incorrect (0). These queries still count toward the denominator, so accuracy and cost reflect the full query set. Please regenerate predictions for these queries and resubmit for a complete evaluation.

---

Evaluation completed by RouterArena automated workflow

[yl231]

Merged via #256, @whyKirbyy. The PR conflicted with main in the price table, so we merged it with current main on our side; your files are unchanged.

You can disregard our Arena-Hard question. Comparing a couple of versions, as you described, doesn't stand in the way.

The leaderboard row will appear in the next ranking update. We're rebuilding the price table and re-pricing every router first, so the posted numbers may differ a little from the 72.48 above.

[yl231]

A correction to my last note, @whyKirbyy: the row needs one more thing before it goes up. Every model call a router makes for a query has to be charged, not just the answer. Corealis's router model (typesafe/jev-1.14) reads every request, so its calls count too.

The evaluator now supports this (#251). Add a router_usage list to each prediction row:

"router_usage": [{"model": "typesafe/jev-1.14", "input_tokens": 640, "output_tokens": 12, "purpose": "routing"}]

output_tokens should count every generated token, reasoning included. uv run python scripts/pricing/compute_cost.py corealis-router shows the total. This PR is already merged, so please send the updated router_inference/predictions/corealis-router.json in a new pull request and comment /evaluate there.

One question: how is typesafe/jev-1.14 served, and is there a published price for it? It needs an entry in model_cost/model_profiles.yaml with a source link. If it runs on your own hardware, let us know its size and precision; we're working on a rule for pricing self-hosted models.

All leaderboard routers were re-priced today at current OpenRouter prices (#262), and Corealis will be priced on the same table.