Reuse provider HTTP clients and manage model context lifetimes

docs/reference/configuration.md

@@ -66,9 +66,9 @@ Transport settings belong to the HTTP client. Setting `httpx2.AsyncClient(trust_
 
 ## HTTP clients
 
-Without `http_client=`, Republic creates and closes a client for each call. Reusing a model alone does not share a connection pool across calls.
+Without `http_client=`, a Provider lazily creates one client on its first request. Models created by the same Provider and model listing share its connection pool. Separate calls to the top-level model factories create separate Providers.
 
-Supply a client to share connections and control its lifetime. Configure timeouts on that client; the provider's `timeout=` option only configures clients Republic creates.
+Supply a client to share connections across Providers and control its lifetime yourself. Configure timeouts on that client; the Provider's `timeout=` option only configures clients Republic creates.
 
 ```python
 import asyncio
@@ -88,8 +88,6 @@ async def main():
 asyncio.run(main())
 ```
 
-Republic leaves a supplied client open. In the example, leaving the application's `async with` block closes it. Exiting a model stream closes that response while leaving the supplied client available for later requests.
-
 This client is used for inference requests and Copilot Plugin token exchange. Codex and Grok token refresh, Copilot device login, and OpenRouter authorization-code exchange create their own auth clients; an injected inference client's transport options do not configure those clients. CLI login methods use the selected CLI's network configuration.
 
 ## Request body merging

docs/reference/provider-api.md

@@ -53,6 +53,68 @@ model = provider.get_model("gpt-6-sol")
 
 Here, `api_base`, `api_key`, and `MyAuth` are supplied by the application. A top-level constructor accepts `provider:model`; a provider instance accepts only the model name. See [configuration](configuration.md) for options and credential precedence, and [authentication](../guides/authentication.md) for API keys and account logins.
 
+### HTTP client lifecycle
+
+A Provider lazily creates one HTTP client and reuses it for requests, streams, model listing, and every model created by that Provider. Separate calls to the top-level model factories create separate Providers.
+
+For applications that manage a long-lived Provider, call `await provider.close()` at shutdown:
+
+```python
+provider = republic.get_provider("openai")
+model = provider.get_model("gpt-6-sol")
+try:
+    response = await model.chat("Hello")
+finally:
+    await provider.close()
+```
+
+Providers also support async context management. Context entries are released on exit, including when the body raises an exception or is cancelled:
+
+```python
+async with republic.get_provider("openai") as provider:
+    first = provider.get_model("gpt-6-sol")
+    second = provider.get_model("gpt-6-luna")
+    await first.chat("Hello")
+    await second.chat("Hello")
+```
+
+Chat, embedding, and decision models support the same pattern. The Provider tracks entered Provider and model contexts and closes only when all of them have exited:
+
+```python
+async with republic.get_model("openai:gpt-6-sol") as model:
+    response = await model.chat("Hello")
+```
+
+Use the same pattern with `get_embedding_model()` and `get_decision_model()`, or enter an existing model with `async with model:`. Entering a Provider or model does not create the HTTP client before its first request. Creating a model without entering its context does not register it as open. Nested contexts for the same model remain open until all its entries have exited.
+
+```python
+provider = republic.get_provider("openai")
+async with provider.get_model("gpt-6-sol") as first:
+    async with provider.get_embedding_model("text-embedding-3-small") as embeddings:
+        vector = await embeddings.embed("Hello")
+    # The Provider remains open while first's context is active.
+    response = await first.chat("Hello")
+# The last model context has exited, so the Provider is now closed.
+```
+
+A Provider context keeps the client open while model contexts enter and exit, so sequential model contexts can share it:
+
+```python
+async with republic.get_provider("openai") as provider:
+    for name in ["gpt-6-sol", "gpt-6-luna"]:
+        async with provider.get_model(name) as model:
+            response = await model.chat("Hello")
+    # The Provider context still keeps the client open here.
+```
+
+Nested Provider contexts follow the same rule: exiting an inner context does not close the client while another Provider or model context is active. Explicit `await provider.close()` closes it immediately, regardless of open contexts.
+
+`provider.close()` is idempotent; `provider.is_closed` reports whether it has been closed. Once closed, all models sharing that Provider reject further requests, and the Provider cannot be reopened. Finish outstanding requests and exit stream contexts before closing it. Exiting a stream context closes only that response and keeps the HTTP client available for subsequent requests.
+
+An internally created client must be used and closed on the event loop that first used it. Constructing the Provider outside an event loop is supported; reusing an active Provider across multiple `asyncio.run()` calls raises `RuntimeError`.
+
+An external client passed through `http_client=` remains caller-owned: closing the Provider leaves that client open. See [HTTP client configuration](configuration.md#http-clients) for transport settings.
+
 ### Tool calls
 
 `tool1` and `tool2` below are `republic.Tool` schemas.

examples/inference.py

@@ -12,96 +12,94 @@ class Profile(pydantic.BaseModel):
 
 
 async def test_decision():
-    model = republic.get_decision_model("openrouter:~typesafe/jev-latest")
-
-    questions: dict[str, republic.decisions.Question] = {
-        "is_urgent": republic.decisions.Noul(
-            instructions="Does this message convey urgency?",
-            criteria={"true": "Explicitly time-sensitive", "false": "No urgency expressed"},
-        ),
-        "department": republic.decisions.Choice(
-            instructions="Which team should handle this?",
-            criteria={
-                "billing": "Payments, invoicing, refunds",
-                "technical": "Bugs, outages, integrations",
-                "sales": "Pricing, upgrades, new accounts",
-            },
-        ),
-        "frustration": republic.decisions.Score(
-            instructions="How frustrated is the customer?",
-            criteria=["Calm", "Frustrated", "Very angry"],
-        ),
-    }
-
-    response = await model.decide(
-        "Help! My payouts have been failing for 3 days.",
-        questions=questions,
-    )
-    print(response.answers)
+    async with republic.get_decision_model("openrouter:~typesafe/jev-latest") as model:
+        questions: dict[str, republic.decisions.Question] = {
+            "is_urgent": republic.decisions.Noul(
+                instructions="Does this message convey urgency?",
+                criteria={"true": "Explicitly time-sensitive", "false": "No urgency expressed"},
+            ),
+            "department": republic.decisions.Choice(
+                instructions="Which team should handle this?",
+                criteria={
+                    "billing": "Payments, invoicing, refunds",
+                    "technical": "Bugs, outages, integrations",
+                    "sales": "Pricing, upgrades, new accounts",
+                },
+            ),
+            "frustration": republic.decisions.Score(
+                instructions="How frustrated is the customer?",
+                criteria=["Calm", "Frustrated", "Very angry"],
+            ),
+        }
+
+        response = await model.decide(
+            "Help! My payouts have been failing for 3 days.",
+            questions=questions,
+        )
+        print(response.answers)
 
 
 async def test_chat():
-    model = republic.get_model("google:gemini-flash-latest")
+    async with republic.get_model("google:gemini-flash-latest") as model:
+        response = await model.chat("What is the capital of France?")
+        print(response.text)
+        print("--------")
 
-    response = await model.chat("What is the capital of France?")
-    print(response.text)
-    print("--------")
-
-    async with model.stream("给出五条今天的新闻", tools=[republic.tools.WebSearch()]) as stream:
-        async for event in stream:
-            if isinstance(event, republic.events.TextDelta):
-                print(event.chunk, end="", flush=True)
-        print()  # Ensure a newline after streaming output
+        async with model.stream("给出五条今天的新闻", tools=[republic.tools.WebSearch()]) as stream:
+            async for event in stream:
+                if isinstance(event, republic.events.TextDelta):
+                    print(event.chunk, end="", flush=True)
+            print()  # Ensure a newline after streaming output
 
 
 async def test_reasoning():
-    model = republic.get_model("deepseek:deepseek-flash")
-    response = await model.chat("Explain the implications of quantum computing.", reasoning_effort="none")
-    print(response.text)
-    assert not response.reasoning, "Expected no reasoning when reasoning_effort is 'none'"  # noqa: S101
-
-    reasoning_head = False
-    answer_head = False
-    async with model.stream("Explain the implications of quantum computing.", reasoning_effort="high") as stream:
-        async for event in stream:
-            if isinstance(event, republic.events.TextDelta):
-                if not answer_head:
-                    print("\n-----------------")
-                    answer_head = True
-                print(event.chunk, end="", flush=True)
-            elif isinstance(event, republic.events.ReasoningDelta):
-                if not reasoning_head:
-                    print("Thinking >")
-                    reasoning_head = True
-                print(event.chunk, end="", flush=True)
-        print()  # Ensure a newline after streaming output
+    async with republic.get_model("deepseek:deepseek-flash") as model:
+        response = await model.chat("Explain the implications of quantum computing.", reasoning_effort="none")
+        print(response.text)
+        assert not response.reasoning, "Expected no reasoning when reasoning_effort is 'none'"  # noqa: S101
+
+        reasoning_head = False
+        answer_head = False
+        async with model.stream("Explain the implications of quantum computing.", reasoning_effort="high") as stream:
+            async for event in stream:
+                if isinstance(event, republic.events.TextDelta):
+                    if not answer_head:
+                        print("\n-----------------")
+                        answer_head = True
+                    print(event.chunk, end="", flush=True)
+                elif isinstance(event, republic.events.ReasoningDelta):
+                    if not reasoning_head:
+                        print("Thinking >")
+                        reasoning_head = True
+                    print(event.chunk, end="", flush=True)
+            print()  # Ensure a newline after streaming output
 
 
 async def test_structured_output():
-    model = republic.get_model("magpie:openrouter/moonshotai/kimi-k3")
-    response = await model.chat("Provide 5 fake users for testing.", output_schema=list[Profile])
-    print(response.output)
-    assert all(isinstance(item, Profile) for item in response.output), (  # noqa: S101
-        "Expected output to be a list of Profile instances"
-    )
+    async with republic.get_model("magpie:openrouter/moonshotai/kimi-k3") as model:
+        response = await model.chat("Provide 5 fake users for testing.", output_schema=list[Profile])
+        print(response.output)
+        assert all(isinstance(item, Profile) for item in response.output), (  # noqa: S101
+            "Expected output to be a list of Profile instances"
+        )
 
 
 async def test_image_generation():
-    model = republic.get_model("google:gemini-3.1-flash-image-preview")
-    response = await model.chat("Generate an image of a futuristic city skyline at sunset.")
-    print(response.text)
-    for i, image in enumerate(response.image_parts):
-        if image.data is None:
-            raise ValueError(f"Image data for image_{i}.png is None")
-        with open(f"image_{i}.png", "wb") as f:
-            f.write(image.data)
+    async with republic.get_model("google:gemini-3.1-flash-image-preview") as model:
+        response = await model.chat("Generate an image of a futuristic city skyline at sunset.")
+        print(response.text)
+        for i, image in enumerate(response.image_parts):
+            if image.data is None:
+                raise ValueError(f"Image data for image_{i}.png is None")
+            with open(f"image_{i}.png", "wb") as f:
+                f.write(image.data)
 
 
 async def test_list_models():
     for provider_name in ["google", "deepseek", "magpie"]:
-        provider = republic.get_provider(provider_name)
-        models = await provider.list_models()
-        print(f"{provider_name.capitalize()} provider has {len(models)} models")
+        async with republic.get_provider(provider_name) as provider:
+            models = await provider.list_models()
+            print(f"{provider_name.capitalize()} provider has {len(models)} models")
 
 
 async def run_all():

src/republic/_models.py

@@ -45,6 +45,15 @@ def __init__(
         self.api_format = api_format
         self.history = history
 
+    async def __aenter__(self) -> Self:
+        self.provider._open_context(self)
+        return self
+
+    async def __aexit__(
+        self, exc_type: type[BaseException] | None, exc: BaseException | None, traceback: TracebackType | None
+    ) -> None:
+        await self.provider._close_context(self)
+
     @overload
     async def chat(
         self, prompt: Input, *, output_schema: None = None, **options: Unpack[ChatOptions]
@@ -211,6 +220,15 @@ def __init__(self, provider: Provider, name: str, api_format: EmbeddingApiFormat
         self.name = name
         self.api_format = api_format
 
+    async def __aenter__(self) -> Self:
+        self.provider._open_context(self)
+        return self
+
+    async def __aexit__(
+        self, exc_type: type[BaseException] | None, exc: BaseException | None, traceback: TracebackType | None
+    ) -> None:
+        await self.provider._close_context(self)
+
     async def embed(self, text: str, *, dimensions: int | None = None) -> EmbeddingResponse:
         """Embed one text; the vector is ``response.vector``."""
         return await self.embed_many([text], dimensions=dimensions)
@@ -238,6 +256,15 @@ def __init__(self, provider: Provider, name: str, api_format: DecisionApiFormat)
         self.name = name
         self.api_format = api_format
 
+    async def __aenter__(self) -> Self:
+        self.provider._open_context(self)
+        return self
+
+    async def __aexit__(
+        self, exc_type: type[BaseException] | None, exc: BaseException | None, traceback: TracebackType | None
+    ) -> None:
+        await self.provider._close_context(self)
+
     async def decide(self, state: JSONValue, *, questions: Mapping[str, Question]) -> DecisionResponse:
         """Answer every question about ``state``. Answers come back under the same ids."""
         request = self.api_format.decision_request(self.name, state, questions)