Skip to content

Pricing: sync at each model's standard price (lab's own price, promotions undone) - #263

Open
yl231 wants to merge 2 commits into
mainfrom
pricing/standard-price
Open

Pricing: sync at each model's standard price (lab's own price, promotions undone)#263
yl231 wants to merge 2 commits into
mainfrom
pricing/standard-price

Conversation

@yl231

@yl231 yl231 commented Oct 31, 2026

Copy link
Copy Markdown
Contributor

For review; not merged. This changes which price the sync takes, and it moves the leaderboard (table below).

Why

Double-checking the 2026-10-31 sync (#250, re-priced in #262): OpenRouter's model-level price, which the sync used, is the price of one endpoint, and it is often not the model's standard price.

  • Gemini 3.7 and 3.8 Flash: every Google endpoint lists discount: 0.5 against Google's announced $1.50 / $7.50 standard price. The model-level price is $0.75 / $3.75, or a regional $0.825 / $4.125.
  • DeepSeek V4 Flash: the model-level $0.091 / $0.182 is one host's 35%-off promotion. Undone, most hosts charge DeepSeek's $0.14 / $0.28.
  • GLM-5.3, GLM-5.3 Flash and Kimi K3: the model-level price is a 4-bit host's price ($0.07 / $1.40, $0.015 / $0.154, $0.192 / $15). The labs themselves charge $1.40 / $4.40, $0.15 / $0.50 and $3 / $15.

The working policy posted on #193 prices a model at its standard published price, regardless of promotions, and the Oct 20 comment there asks for exactly this rule for introductory prices.

The rule

For every synced profile, the sync reads OpenRouter's per-provider endpoints and takes:

  1. Standard tier only. Flex, Priority and batch tiers are excluded, and only active endpoints count when any are.
  2. Promotions undone. An endpoint's price is divided by 1 − discount.
  3. The lab's own price, when the lab sells the model on OpenRouter (Google, OpenAI, Anthropic, Z.AI, Moonshot AI, Mistral, xAI, …). Its lowest standard price is used, so Google's global price rather than a regional one.
  4. Otherwise the median over hosts, without 4-bit hosts when higher-precision ones exist.

Missing data: a model whose endpoints can't be read keeps its price and is listed in the report.
Snapshot: records each price's basis.
Report: flags prices that double or halve.

Prices that change (OpenRouter data of 2026-10-31)

Model OpenRouter id Old New Basis
deepseek/deepseek-chat deepseek/deepseek-chat 0.2574 / 1.0287 0.286 / 1.143 median of 1 host(s), 10% promotion undone
deepseek/deepseek-v3.2 ⚠️ deepseek/deepseek-v3.2 0.26 / 0.38 0.295 / 0.88 median of 8 host(s), 28% promotion undone
deepseek/deepseek-v4-flash deepseek/deepseek-v4-flash 0.091 / 0.182 0.14 / 0.28 median of 13 host(s), 72% promotion undone
deepseek/deepseek-v4-flash-0731 ⚠️ deepseek/deepseek-v4-flash-0731 0.119 / 0.238 0.176 / 0.528 median of 19 host(s), 90% promotion undone
deepseek/deepseek-v4.1-flash ⚠️ deepseek/deepseek-v4.1-flash 0.045 / 0.5 0.3 / 1.2 median of 30 host(s), 85% promotion undone
google/gemini-3.7-flash ⚠️ google/gemini-3.7-flash 0.75 / 3.75 1.5 / 7.5 Google AI Studio (the lab's own price), 50% promotion undone
google/gemini-3.8-flash google/gemini-3.8-flash 0.825 / 4.125 1.5 / 7.5 Google AI Studio (the lab's own price), 50% promotion undone
google/gemma-4-31b-it google/gemma-4-31b-it 0.2 / 0.4 0.15 / 0.4 median of 7 host(s), 5% promotion undone
inclusionai/ling-3.0-flash ⚠️ inclusionai/ling-3.0-flash 0.021 / 0.063 0.06 / 0.18 median of 1 host(s), 65% promotion undone
meta-llama/llama-3.3-70b-instruct ⚠️ meta-llama/llama-3.3-70b-instruct 0.1 / 0.32 0.4415 / 0.715 median of 10 host(s), 25% promotion undone
moonshotai/kimi-k2.5 moonshotai/kimi-k2.5 0.5 / 2.5 0.6 / 3 median of 3 host(s), 5% promotion undone
moonshotai/kimi-k2.6 moonshotai/kimi-k2.6 0.65 / 3.41 0.95 / 4 Moonshot AI (the lab's own price)
moonshotai/kimi-k3 ⚠️ moonshotai/kimi-k3 0.192 / 15 3 / 15 Moonshot AI (the lab's own price)
openai/gpt-4.1-nano openai/gpt-4.1-nano 0.1 / 0.4 0.105 / 0.42 median of 2 host(s)
openai/gpt-oss-20b ⚠️ openai/gpt-oss-20b 0.018 / 0.09 0.04 / 0.15 median of 7 host(s)
qwen/qwen3-235b-a22b-2507 qwen/qwen3-235b-a22b-2507 0.09 / 0.55 0.145 / 0.775 median of 4 host(s), 75% promotion undone
qwen/qwen3-30b-a3b-instruct-2507 qwen/qwen3-30b-a3b-instruct-2507 0.1 / 0.3 0.09 / 0.3 median of 3 host(s)
qwen/qwen3-coder qwen/qwen3-coder 0.3 / 1 0.35 / 1.5 median of 1 host(s)
qwen/qwen3-coder-30b-a3b-instruct qwen/qwen3-coder-30b-a3b-instruct 0.07 / 0.28 0.11 / 0.44 median of 2 host(s)
qwen/qwen3-next-80b-a3b-instruct qwen/qwen3-next-80b-a3b-instruct 0.09 / 1.1 0.095 / 1.1 median of 2 host(s)
qwen/qwen3.5-9b qwen/qwen3.5-9b 0.17 / 0.25 0.1 / 0.15 median of 5 host(s)
upstage/solar-pro4 ⚠️ upstage/solar-pro4 0.15 / 0.6 0.3 / 1.2 Upstage (the lab's own price), 50% promotion undone
z-ai/glm-4.5-air z-ai/glm-4.5-air 0.13 / 0.85 0.2 / 1.1 Z.AI (the lab's own price)
z-ai/glm-4.6 z-ai/glm-4.6 0.5 / 2 0.6 / 2.2 Z.AI (the lab's own price)
z-ai/glm-5.3 ⚠️ z-ai/glm-5.3 0.0699 / 1.4 1.4 / 4.4 Z.AI (the lab's own price)
z-ai/glm-5.3-flash ⚠️ z-ai/glm-5.3-flash 0.0149 / 0.1543 0.15 / 0.5 Z.AI (the lab's own price)

⚠️ marks a price that at least doubled or halved.

Leaderboard effect once re-priced (reprice.py --all)

Router Arena Cost / 1K queries Accuracy
divyam-router 77.19 → 76.22 $0.6643 → $1.0986 80.83
r2-reasonometry-router 76.16 → 76.16 $0.4841 → $0.4842 78.97
sqwish-fvcta8 76.34 → 76.10 $0.6875 → $0.7845 79.88
KT-ModelRouter 76.08 → 76.09 $0.3354 → $0.3317 78.24
llm-router-g3 75.77 → 75.77 $0.4905 → $0.4905 78.54
PSAI-Auto-Router 75.58 → 75.55 $0.4679 → $0.4790 78.23
BizRouter 77.08 → 75.47 $1.0023 → $2.0040 81.66
cross-router 75.25 → 75.14 $0.5923 → $0.6345 78.27
nadir-caliper 74.63 → 74.51 $0.2516 → $0.2769 76.14
vllm-sr 74.24 → 74.15 $0.6671 → $0.7038 77.30
BARouter 73.67 → 73.44 $0.5369 → $0.6221 76.22
nuroutev2-artemis 73.63 → 73.02 $0.5414 → $0.7949 76.19
orcarouter-adaptive 72.48 → 72.47 $0.8260 → $0.8337 75.63
nadir-cascade-v2 72.25 → 72.20 $0.6725 → $0.6938 74.96
hybrid-router 71.20 → 70.86 $0.1260 → $0.1811 71.46
azure-model-router 70.37 → 70.37 $0.7521 → $0.7521 72.93
chuzom-solo-v32 70.64 → 70.25 $0.1005 → $0.1551 70.62
weave-router 70.00 → 70.00 $3.2082 → $3.2082 76.22
AsiaInfo-Router 70.00 → 69.98 $2.8646 → $2.8831 75.82
auto_router 69.80 → 69.75 $0.2056 → $0.2169 70.42
cruq-router 69.89 → 69.40 $0.4121 → $0.6019 71.41
lynkr 67.97 → 67.95 $0.3069 → $0.3103 68.83
chayan 63.70 → 63.70 $0.6135 → $0.6135 64.87

Most of the change comes from Gemini 3.7 and 3.8 Flash doubling to Google's standard price.

  • BizRouter (more than 99% of its cost on Gemini 3.7 Flash) goes from 77.08 to 75.47.
  • Divyam goes from 77.19 to 76.22.
  • Google's standard price applies from January 1 anyway.

Open question

DeepSeek's own prices. DeepSeek sells its models through its own API but not through OpenRouter, so its models fall back to the hosts' median:

  • V4 Flash: the median matches DeepSeek's $0.14 / $0.28.
  • V3.2: the median gives $0.295 / $0.88, against DeepSeek's own $0.28 / $0.42.

Pinning a lab's published price by hand (sync: false plus a source link) would fix such cases.

Testing

  • tests/test_price_sync.py covers the lab's price with the promotion undone, the median of full-precision hosts, Flex and inactive endpoints, missing endpoint data, and the snapshot basis. All 44 pricing tests pass.
  • pre-commit passes.
  • The re-sync uses the saved 2026-10-31 OpenRouter responses, so the run is reproducible.

🤖 Generated with Claude Code

yl231 and others added 2 commits October 31, 2026 17:40
@yl231 @claude
Price sync: use each model's standard price, not OpenRouter's model-l…
5f5f6b6 
…evel price

OpenRouter's model-level price is one endpoint's price, and often not the
model's standard price: Gemini 3.7/3.8 Flash show a 50% introductory price
($0.75/$3.75 or a regional $0.825/$4.125) against Google's announced
$1.50/$7.50; DeepSeek V4 Flash shows a host's 35%-off promotion; GLM-5.3
and Kimi K3 show a 4-bit host's price instead of the lab's.

The sync now reads each model's per-provider endpoints and takes:
- standard-tier endpoints only (no Flex, Priority or batch), active ones
  when any are;
- each endpoint's promotional discount undone (price / (1 - discount));
- the lab's own lowest standard price when the lab sells the model on
  OpenRouter;
- otherwise the median over hosts, without 4-bit quantizations when
  higher-precision hosts exist.

A model whose endpoints can't be read keeps its price and is listed in
the report. The snapshot records each price's basis. Prices that double or
halve are flagged in the report.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@yl231 @claude
Re-sync prices with the standard-price rule (OpenRouter data of 2026-…
34c48af 
…10-31)

26 prices change, e.g. Gemini 3.7/3.8 Flash 0.75-0.825/3.75-4.125 ->
1.50/7.50 (Google's standard price), DeepSeek V4 Flash 0.091/0.182 ->
0.14/0.28, GLM-5.3 0.07/1.40 -> 1.40/4.40 and GLM-5.3 Flash
0.015/0.154 -> 0.15/0.50 (Z.AI's price), Kimi K3 0.192/15 -> 3/15
(Moonshot's price). The 2026-10-31 snapshot is rewritten with the prices
and their basis. The leaderboard is not re-priced here.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Oct 31, 2026
Open
Open
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@yl231