[yl231]

For review; not merged. The quantization factors are a policy choice; the data behind them is below.

A rule for pricing models that a router runs itself, on-device or self-hosted (#193, #198). It follows the working policy in #193: price the model, not the submitter's bill.

The rule

A profile in model_profiles.yaml marks a local model:

"lemonade/deepseek-v4-flash":
  price: {input: 0.112, output: 0.224}
  local: {base: "deepseek/deepseek-v4-flash", bits: 2}   # the weights of a listed model
  sync: false
"some/unlisted-27b-q4":
  price: {input: 0.156, output: 0.488}
  local: {active_params_b: 27, bits: 4}                 # no listed model has these weights
  sync: false

Price:

1. Same weights as a listed model: that model's price from the table, so a local copy is never cheaper than the same model on the board just because it runs locally.
2. Otherwise: the market price of an open-weight model of that size, from a size curve.

Either way, it's multiplied by a quantization factor for the served weights: ×1.0 at 8 bits or more, ×0.8 below.

The data (model_cost/local_pricing.yaml)

The data is 103 open-weight models on OpenRouter with a known parameter count, at standard prices: standard tier, promotions undone, median over hosts at 8 bits or more.

Size curve:

• input = 0.0669 × B^0.325;
• output = 0.2153 × B^0.316;
• B is active parameters in billions. Price grows slowly with size, and the spread around the curve is wide (about ×2.3 and ×2.8).

Active params	0.35B	3B	8B	27B	70B	235B
Input / output (USD per 1M)	0.048 / 0.155	0.096 / 0.305	0.131 / 0.415	0.195 / 0.610	0.266 / 0.824	0.395 / 1.21

Quantization, comparing hosts of the same model:

• 8-bit vs 16-bit: 1.00×, across 23 models;
• 4-bit vs 8/16-bit: 0.80× input and 0.83× output, across 41 models.

There's no market data below 4 bits, so 2- and 3-bit weights get the 4-bit factor. This is smaller than the ×0.75 (8-bit) and ×0.5 (4-bit) I had proposed earlier, because the market doesn't discount quantized hosting that much.

What it gives the pending local submission (#198, Lemonade)

Model	Rule	Price	Lemonade's entry
lemonade/deepseek-v4-flash (2-bit)	DeepSeek V4 Flash × 0.8	0.112 / 0.224 at the standard price (#263); 0.0728 / 0.146 at today's table	0.02 / 0.04
lemonade/Qwen3.8-27B-GGUF-UD-Q4_K_XL	Qwen3.8-27B is listed: Alibaba's 0.425 / 2.55 × 0.8	0.34 / 2.04
lemonade/LFM2.5-Embedding-350M	not listed: size curve at 0.35B	0.0476 / 0.155

Tools

• python scripts/pricing/local_price.py --active-params-b 27 --bits 4 (or --base <id> --bits N) prints a price and how it was derived.
• --write updates every local profile and model_cost.json.
• --check runs as a pre-commit hook and fails when a local price is out of date, e.g. after a sync moves its base model's price.
• scripts/pricing/fit_local_rule.py refits the curve and the quantization ratios from live OpenRouter and Hugging Face data. It reproduces the numbers above.
• The README's submission section explains how to add a local model.

Testing

• tests/test_local_price.py covers the factors, base vs size curve, rejected specs, and --check/--write end to end. All 48 pricing tests pass.
• pre-commit passes.

🤖 Generated with Claude Code