[yl231]

Part of the pricing rework, after #218 and #250 (both merged); the re-pricing tool (#261) builds on it. It implements the plan for routers that call LLMs themselves (#198, #209), so that every such call is charged.

What it adds

• router_usage on a prediction row (optional): one record per LLM call the router itself made to decide, whether a routing model, a classifier, voters, a critic, or the earlier steps of a cascade:

  "router_usage": [{"model": "openai/gpt-4o-mini", "input_tokens": 812, "output_tokens": 37, "purpose": "routing"}]

  output_tokens counts every generated token, reasoning included.
• Evaluator: a row's cost is the answer's cost plus its router calls, each at its own model's price; a model without a profile pays the table's maximum. The router part is also stored as router_cost. The per-row logic moves into compute_row_cost() in llm_evaluation/run.py, unchanged for the answer (it still prefers model_used when priced, Leaderboard integrity: retired Grok 4.1 Fast slug silently resolved to Grok 4.3 at the old price #166).
• Checker: validates router_usage, and its models join the missing-price warning.
• Toolkit: uv run python scripts/pricing/compute_cost.py <router> prints answer, router-call and total cost per 1K queries, plus any model charged the maximum. --per-row out.csv writes a per-row breakdown. It calls the evaluator's own compute_row_cost(), so both agree by construction, and it grades nothing, so it runs offline in seconds. On KT-ModelRouter it gives $0.2698/1K, matching the board's $0.27.
• README: a note in the submission steps.
• Tests: tests/test_router_usage.py; with the Model price profiles: one entry per model, exact price matching, max price for unmatched models #218 and Pricing: sync model prices with OpenRouter (manual), applied with the 2026-10-31 list #250 tests, 25 pass.

Not in this PR

• Local models used as routers or answerers are priced by the local-model rule, which comes in a separate PR.
• Existing submissions that already reported router calls in their own formats (Compass's cascade_accounting, IQ Routing's note, dalpha's classifier) would move to router_usage when they resubmit or are re-priced.

🤖 Generated with Claude Code