Add Novita as a model gateway backend (#102)

* Add Novita as a model gateway backend

Novita hosts open-weight models behind its own OpenAI-compatible
endpoint (api.novita.ai/openai), reached with a single NOVITA_API_KEY.
Model ids are already author/slug (deepseek/deepseek-r1), the same
shape OpenRouter uses, so no special-casing is needed in vendor().

Follows the existing openrouter/openai/ollama backends: a thin
subclass of OpenAICompatChatModel, credentials resolved through the
same env/.env lookup in config.py, and a new branch in the registry's
get_model(). Like OpenAI, Novita reports no per-call cost, so calls
are counted as unpriced unless a rate card is supplied.

* Recommend Novita's current flagship models

The models listed for Novita were older ids that no longer reflect what
the platform leads with. Point the recommendations at the three current
flagships instead, each verified against api.novita.ai:

  moonshotai/kimi-k3              1M context, native vision
  zai-org/glm-5.2                 1M context, long-horizon agentic work
  deepseek/deepseek-v4-flash-0731 1M context, cheapest of the three

Context windows, output limits, input modalities and pricing were taken
from the live /openai/v1/models response rather than carried over.

* Fix column alignment for novita example in models cookbook

---------

Co-authored-by: jax-novita <jax-novita@users.noreply.github.com>

Author: jax-novita

docs/cookbook/02-models.md

@@ -24,6 +24,7 @@ for spec in (
     "claude-cli/claude-sonnet-5",
     "openrouter/anthropic/claude-haiku-4.5",
     "openai/gpt-4o-mini",
+    "novita/moonshotai/kimi-k3",
     "ollama/llama3.1",
     "mock/anything",
 ):
@@ -38,18 +39,20 @@ print(model.invoke("say hi").content)
 {'spec': 'claude-cli/claude-sonnet-5', 'backend': 'claude-cli', 'model': 'claude-sonnet-5'}
 {'spec': 'openrouter/anthropic/claude-haiku-4.5', 'backend': 'openrouter', 'model': 'anthropic/claude-haiku-4.5'}
 {'spec': 'openai/gpt-4o-mini', 'backend': 'openai', 'model': 'gpt-4o-mini'}
+{'spec': 'novita/moonshotai/kimi-k3', 'backend': 'novita', 'model': 'moonshotai/kimi-k3'}
 {'spec': 'ollama/llama3.1', 'backend': 'ollama', 'model': 'llama3.1'}
 {'spec': 'mock/anything', 'backend': 'mock', 'model': 'anything'}
 hello from a scripted model
 ```
 
-There are five backends:
+There are six backends:
 
 | backend | credential | what it is for |
 | --- | --- | --- |
 | `claude-cli` | a Claude subscription, no API key | text completion on quota you already pay for |
 | `openrouter` | `OPENROUTER_API_KEY` | one key, most vendors, per-call cost in the response |
 | `openai` | `OPENAI_API_KEY` | the OpenAI API directly, or any endpoint via `OPENAI_BASE_URL` |
+| `novita` | `NOVITA_API_KEY` | Novita's own endpoint, token counts but no per-call cost |
 | `ollama` | none — a local server | models on your own machine, free and offline |
 | `mock` | none | a scripted test double |
 
@@ -63,8 +66,8 @@ needs it. Asking for `claude-cli` therefore does not require `langchain-openai`,
 for `openrouter` does not require the Claude CLI to be installed. A missing optional
 dependency fails for the backend that wanted it and nothing else.
 
-`openrouter`, `openai` and `ollama` all speak the OpenAI wire format and share one base
-class, so they behave identically on everything except money and routing: same
+`openrouter`, `openai`, `novita` and `ollama` all speak the OpenAI wire format and share one
+base class, so they behave identically on everything except money and routing: same
 `bind_tools`, same `with_structured_output`, same streaming and async, same retry policy,
 same usage envelope.
 
@@ -93,7 +96,7 @@ except UnknownBackendError as exc:
 ('openrouter', 'openai/gpt-4o-mini:floor')
 ('claude-cli', 'anthropic/claude-haiku-4.5')
 ('openai', 'gpt-4o-mini')
-UnknownBackendError: unknown backend 'opnerouter' in spec 'opnerouter/openai/gpt-4o-mini'; expected one of: claude-cli, openrouter, openai, ollama, mock — or a bare model name for the claude-cli default
+UnknownBackendError: unknown backend 'opnerouter' in spec 'opnerouter/openai/gpt-4o-mini'; expected one of: claude-cli, openrouter, openai, novita, ollama, mock — or a bare model name for the claude-cli default
 ```
 
 Only the first segment is a backend, because OpenRouter model ids are themselves
@@ -122,16 +125,18 @@ model through the broker is still `openrouter/openai/gpt-4o-mini`.
 <!-- verified: cli -->
 ```console
 $ grapharc models
-backends: claude-cli, openrouter, openai, ollama, mock
+backends: claude-cli, openrouter, openai, novita, ollama, mock
 openrouter key: <unset>
 openai key: <unset>
+novita key: <unset>
 ollama url: http://localhost:11434/v1
 
 examples:
   claude-cli/claude-sonnet-5              subscription, no API key
   openrouter/anthropic/claude-haiku-4.5   many providers, one key
   openrouter/openai/gpt-4o-mini:floor     cheapest provider for that model
   openai/gpt-4o-mini                      the OpenAI API directly, your key
+  novita/moonshotai/kimi-k3               Novita's own endpoint, your key
   ollama/llama3.1                         a local server, no key and no bill
 
 grapharc models --check  probes which of these this machine can use
@@ -165,6 +170,8 @@ openrouter   unusable  no API key (set OPENROUTER_API_KEY, or add one to .env)
                        credential: <unset>
 openai       unusable  no API key (set OPENAI_API_KEY, or add one to .env)
                        credential: <unset>
+novita       unusable  no API key (set NOVITA_API_KEY, or add one to .env)
+                       credential: <unset>
 ollama       usable    local server at http://localhost:11434/v1
                        credential: none needed (local server)
 mock         usable    scripted test double; never reaches a provider

grapharc/cli/main.py

@@ -463,6 +463,7 @@ def _cmd_approve(args: argparse.Namespace) -> int:
 def _cmd_models(args: argparse.Namespace) -> int:
     from grapharc.gateway import (
         describe,
+        novita_api_key,
         ollama_base_url,
         openai_api_key,
         openrouter_api_key,
@@ -511,6 +512,7 @@ def _cmd_models(args: argparse.Namespace) -> int:
         "openrouter/anthropic/claude-haiku-4.5": "many providers, one key",
         "openrouter/openai/gpt-4o-mini:floor": "cheapest provider for that model",
         "openai/gpt-4o-mini": "the OpenAI API directly, your key",
+        "novita/moonshotai/kimi-k3": "Novita's own endpoint, your key",
         "ollama/llama3.1": "a local server, no key and no bill",
     }
     payload = {
@@ -519,6 +521,7 @@ def _cmd_models(args: argparse.Namespace) -> int:
         "backends": list(BACKENDS),
         "openrouter_key": redact(openrouter_api_key()),
         "openai_key": redact(openai_api_key()),
+        "novita_key": redact(novita_api_key()),
         # An address, not a secret: it is printed whole, and it is where a
         # request would go rather than proof that anything is listening.
         "ollama_base_url": ollama_base_url(),
@@ -530,6 +533,7 @@ def _cmd_models(args: argparse.Namespace) -> int:
         style.kv("backends", ", ".join(BACKENDS)),
         style.kv("openrouter key", redact(openrouter_api_key())),
         style.kv("openai key", redact(openai_api_key())),
+        style.kv("novita key", redact(novita_api_key())),
         style.kv("ollama url", ollama_base_url(), tint=style.accent),
         "",
         style.heading("examples:"),

grapharc/cli/probe.py

@@ -95,6 +95,26 @@ def _probe_openai() -> dict[str, Any]:
     }
 
 
+def _probe_novita() -> dict[str, Any]:
+    from grapharc.gateway import novita_api_key, redact
+
+    key = novita_api_key()
+    has_dependency = importlib.util.find_spec("langchain_openai") is not None
+    missing = []
+    if not key:
+        missing.append("no API key (set NOVITA_API_KEY, or add one to .env)")
+    if not has_dependency:
+        missing.append("langchain-openai not installed (uv sync --extra novita)")
+    return {
+        "backend": "novita",
+        "kind": KIND_PROVIDER,
+        "usable": bool(key) and has_dependency,
+        "credential": redact(key),
+        "detail": "; ".join(missing) or "api key configured and langchain-openai installed",
+        "checked": "credential presence only; no request was sent to api.novita.ai",
+    }
+
+
 def _probe_ollama() -> dict[str, Any]:
     from grapharc.gateway import ollama_base_url
 
@@ -152,6 +172,7 @@ def probe_backends(*, claude_path: str = "claude") -> list[dict[str, Any]]:
         "claude-cli": lambda: _probe_claude_cli(claude_path),
         "openrouter": _probe_openrouter,
         "openai": _probe_openai,
+        "novita": _probe_novita,
         "ollama": _probe_ollama,
         "mock": _probe_mock,
     }

grapharc/gateway/__init__.py

@@ -6,6 +6,7 @@
     get_model("claude-cli/claude-sonnet-5")             # subscription, no API key
     get_model("openrouter/anthropic/claude-sonnet-4.5") # many providers, one key
     get_model("openai/gpt-4o-mini")                     # OPENAI_API_KEY
+    get_model("novita/moonshotai/kimi-k3")            # NOVITA_API_KEY
     get_model("ollama/llama3.1")                        # local server, no key
     get_model("mock/x", responses=[...])                # deterministic tests
 
@@ -16,12 +17,13 @@
     get_model(spec, cost_ceiling_usd=0.25)                     # raises when passed
     get_model(spec, spend=shared_meter)                        # one ceiling, many models
 
-The three OpenAI-wire backends (`openrouter`, `openai`, `ollama`) are imported
-lazily — they need `langchain-openai`, which is an optional extra.
+The four OpenAI-wire backends (`openrouter`, `openai`, `novita`, `ollama`) are
+imported lazily — they need `langchain-openai`, which is an optional extra.
 """
 
 from grapharc.gateway.claude_cli import ClaudeCodeCLIChatModel
 from grapharc.gateway.config import (
+    novita_api_key,
     ollama_api_key,
     ollama_base_url,
     openai_api_key,
@@ -66,6 +68,7 @@
     "different_providers",
     "get_model",
     "is_transient",
+    "novita_api_key",
     "ollama_api_key",
     "ollama_base_url",
     "openai_api_key",
@@ -85,6 +88,8 @@
     "OpenRouterError": "openrouter",
     "OpenAIChatModel": "openai",
     "OpenAIError": "openai",
+    "NovitaChatModel": "novita",
+    "NovitaError": "novita",
     "OllamaChatModel": "ollama",
     "OllamaError": "ollama",
 }

grapharc/gateway/config.py

@@ -20,14 +20,17 @@
 Secrets are returned, never logged. Anything that renders a config for humans
 goes through `redact`.
 
-Three key-holding backends, plus one that usually holds none:
+Four key-holding backends, plus one that usually holds none:
 
 - **OpenRouter** — `OPENROUTER_API_KEY`.
 - **OpenAI** — `OPENAI_API_KEY`, optionally with `OPENAI_BASE_URL` for an
   Azure-style or proxied endpoint. `langchain-openai` reads `OPENAI_API_KEY`
   from the process environment on its own; going through here as well is what
   adds `.env` support, the alternate spellings, and a failure that names the
   variable instead of surfacing an SDK error.
+- **Novita** — `NOVITA_API_KEY`. The endpoint is fixed
+  (`grapharc.gateway.novita.NOVITA_BASE_URL`), so unlike OpenAI there is no
+  base-url override to resolve here.
 - **Ollama** — no credential by default: it is a server on your own machine.
   What it needs is an address, so `ollama_base_url()` always returns one
   (`OLLAMA_HOST` / `OLLAMA_BASE_URL`, else localhost). `OLLAMA_API_KEY` exists
@@ -56,6 +59,13 @@
     "openai_api_key",
 )
 
+NOVITA_KEYS = (
+    "NOVITA_API_KEY",
+    "NOVITA_KEY",
+    "novita-api-key",
+    "novita_api_key",
+)
+
 # `OPENAI_API_BASE` is the older spelling and is still what a lot of tooling
 # sets; both are accepted, the current one first.
 OPENAI_BASE_URL_KEYS = (
@@ -133,6 +143,10 @@ def openai_api_key(*, env_file: Path | None = None) -> str | None:
     return get_secret(OPENAI_KEYS, env_file=env_file)
 
 
+def novita_api_key(*, env_file: Path | None = None) -> str | None:
+    return get_secret(NOVITA_KEYS, env_file=env_file)
+
+
 def openai_base_url(*, env_file: Path | None = None) -> str | None:
     """An endpoint override, or None for api.openai.com.
 

grapharc/gateway/novita.py

@@ -0,0 +1,59 @@
+"""Novita backend — a GPU cloud hosting open-weight models, one key.
+
+`novita/moonshotai/kimi-k3` reaches Novita's own OpenAI-compatible endpoint
+(`https://api.novita.ai/openai`), not api.openai.com, so this builds on
+`OpenAICompatChatModel` the same way `openrouter.py` and `ollama.py` do rather
+than on `openai.py`: the endpoint is fixed, not an override of OpenAI's own.
+
+Model ids on Novita are themselves `author/slug` — `moonshotai/kimi-k3`,
+`zai-org/glm-5.2` — the same shape OpenRouter uses, so `vendor()` in
+`registry.py` already reads the right author off a Novita spec with no
+backend-specific handling: `BACKEND_VENDOR` stays absent for `novita`, exactly
+as it is absent for `openrouter`.
+
+**Novita reports no per-call cost.** Unlike OpenRouter, the chat-completions
+response carries token counts and nothing else, so this backend is the OpenAI
+one in that respect: `_provider_cost` is the base class's `None`, and
+`cost_ceiling_usd` counts calls in `SpendMeter.unpriced_calls` unless a caller
+supplies `price_per_million=`.
+"""
+
+from __future__ import annotations
+
+from typing import Any
+
+from grapharc.gateway.config import novita_api_key
+from grapharc.gateway.openai_compat import OpenAICompatChatModel
+
+NOVITA_BASE_URL = "https://api.novita.ai/openai"
+
+
+class NovitaError(Exception):
+    """The Novita backend could not be constructed or used."""
+
+
+class NovitaChatModel(OpenAICompatChatModel):
+    """A LangChain chat model over Novita's OpenAI-compatible endpoint."""
+
+    def __init__(self, model: str, /, **kwargs: Any) -> None:
+        api_key = kwargs.pop("api_key", None) or novita_api_key()
+        if not api_key:
+            raise NovitaError(
+                "No Novita API key found. Set NOVITA_API_KEY in the environment, "
+                "or add one of NOVITA_API_KEY / novita-api-key to a .env file."
+            )
+        # One retry layer, not two — same reasoning as the OpenRouter backend.
+        kwargs.setdefault("max_retries", 0)
+        super().__init__(
+            model=model,
+            api_key=api_key,
+            base_url=kwargs.pop("base_url", None) or NOVITA_BASE_URL,
+            **kwargs,
+        )
+
+    @property
+    def _llm_type(self) -> str:
+        return "grapharc-novita"
+
+
+__all__ = ["NOVITA_BASE_URL", "NovitaChatModel", "NovitaError"]

grapharc/gateway/registry.py

@@ -7,6 +7,7 @@
     openrouter/anthropic/claude-sonnet-4.5  -> OpenRouter
     openrouter/openai/gpt-4o:floor          -> OpenRouter, cheapest provider
     openai/gpt-4o-mini                      -> OpenAI directly (OPENAI_API_KEY)
+    novita/moonshotai/kimi-k3             -> Novita (NOVITA_API_KEY)
     ollama/llama3.1                         -> a local Ollama server, no key
     mock/anything                           -> scripted test double
 
@@ -28,7 +29,7 @@ class UnknownBackendError(Exception):
     """The spec named a backend that is not registered."""
 
 
-BACKENDS = ("claude-cli", "openrouter", "openai", "ollama", "mock")
+BACKENDS = ("claude-cli", "openrouter", "openai", "novita", "ollama", "mock")
 
 # Authors that appear in OpenRouter model ids. A spec starting with one of
 # these is a model name, not a mistyped backend — `anthropic/claude-haiku-4.5`
@@ -71,6 +72,7 @@ class UnknownBackendError(Exception):
 _BARE_BACKEND_EXAMPLE = {
     "openrouter": "openrouter/anthropic/claude-sonnet-4.5",
     "openai": "openai/gpt-4o-mini",
+    "novita": "novita/moonshotai/kimi-k3",
     "ollama": "ollama/llama3.1",
 }
 
@@ -160,6 +162,17 @@ def get_model(spec: str, **kwargs: Any) -> BaseChatModel:
 
         return OpenAIChatModel(model, **kwargs)
 
+    if backend == "novita":
+        try:
+            from grapharc.gateway.novita import NovitaChatModel
+        except ImportError as exc:  # pragma: no cover - depends on install extras
+            raise UnknownBackendError(
+                "The Novita backend needs langchain-openai. "
+                "Install it with: uv sync --extra novita"
+            ) from exc
+
+        return NovitaChatModel(model, **kwargs)
+
     if backend == "ollama":
         try:
             from grapharc.gateway.ollama import OllamaChatModel

pyproject.toml

@@ -71,6 +71,12 @@ openrouter = [
 openai = [
     "langchain-openai>=0.2",
 ]
+# Imported by grapharc/gateway/novita.py. Novita speaks the OpenAI wire format
+# against its own endpoint, so this needs the same client and no Novita-specific
+# package.
+novita = [
+    "langchain-openai>=0.2",
+]
 # Imported by grapharc/gateway/ollama.py. Ollama speaks the OpenAI wire format,
 # so the local backend needs the same client and no ollama-specific package.
 ollama = [
@@ -115,7 +121,7 @@ slack = [
 ]
 # Everything above. Self-referential so it cannot drift out of sync.
 all = [
-    "grapharc[api,ladybug,mcp,ollama,openai,openrouter,otel,server,slack]",
+    "grapharc[api,ladybug,mcp,novita,ollama,openai,openrouter,otel,server,slack]",
 ]
 
 [dependency-groups]

tests/test_cli.py

@@ -438,6 +438,7 @@ def test_models_check_exits_one_when_nothing_is_configured(monkeypatch, capsys):
     for name in (
         "OPENROUTER_API_KEY", "OPENROUTER_KEY", "open-router-api-key",
         "OPENAI_API_KEY", "OPENAI_KEY", "openai-api-key",
+        "NOVITA_API_KEY", "NOVITA_KEY", "novita-api-key",
         "OLLAMA_HOST", "OLLAMA_BASE_URL",
     ):
         monkeypatch.delenv(name, raising=False)
@@ -452,10 +453,11 @@ def test_models_check_exits_one_when_nothing_is_configured(monkeypatch, capsys):
         "claude-cli": False,
         "openrouter": False,
         "openai": False,
+        "novita": False,
         "ollama": False,
         "mock": True,
     }
-    for backend in ("openrouter", "openai"):
+    for backend in ("openrouter", "openai", "novita"):
         assert next(b for b in payload["backends"] if b["backend"] == backend)[
             "credential"
         ] == "<unset>"