adk-local-models: a direct local inference adapter for Google ADK

[AubinGil]

Hi everyone! I've been experimenting with Google ADK agents using locally hosted models, and built adk-local-models, an Apache-2.0 Python adapter implementing ADK's BaseLlm interface.

ADK already supports local OpenAI-compatible servers through its native google.adk.integrations.openai.OpenAILlm class, using the OpenAI SDK, as well as through LiteLLM. The native integration is a good default for most users. I wanted a small, explicit adapter using ADK's existing httpx dependency, with neither LiteLLM nor the OpenAI SDK and with specific response and recording behavior.

The adapter maps ADK's text and function-tool requests to /v1/chat/completions and maps replies back to ADK responses. It provides:

• vllm, ollama and llama_server profiles with validation for documented profile knobs.
• Text, streaming and function calling, with tool calls emitted only in the final streamed response.
• HTTP, transport and malformed-reply errors returned as ADK error responses, plus truncated-stream detection.
• SHA-256 hashes of exact request/reply entity bytes, a completeness flag and cached advertised server identity in response metadata. Production responses carry hashes, not retained raw bytes; identity is not an attestation.

Pass the instance as LlmAgent(model=llm, ...) to use it with ADK's Runner. Streamed final responses contain the complete output, so don't append them to accumulated partial text.

Evidence and scope: Live tests covered text and function calls, plain and streamed, and Runner round trips on one model per backend: vLLM with nemotron-3.5-bolt-e4b, Ollama with gemma4:e4b, and llama-server with LFM3-1.7B Q4_K_M on llama.cpp 0.8.0 (b11960), using --jinja --reasoning off. The native comparison exercised both adapters live on vLLM and Ollama; its llama-server comparison was fixture-only. Later llama-server runs validated this adapter live. Failure cases, including truncated streams, were checked with fake-server fixtures. This is compatibility evidence for those configurations, not a benchmark or a guarantee for every model/template.

Limitations: Text and function tools only; no images, audio, live bidirectional API, strict output-schema enforcement, reasoning-content replay or reasoning-token usage mapping. JSON mode maps to json_object. llama-server tools require --jinja and a suitable template; on the tested build, reasoning_format="none" with JSON mode produces HTTP 400, so the adapter rejects that combination before I/O. It never configures or manages servers.

Native OpenAILlm offers broader image, strict-schema and reasoning support; those broader paths were reviewed in source, not exercised in the comparison. Its HTTP client can also be configured to capture bytes. This adapter focuses on the particular error, streaming, profile-validation and metadata contracts above. "Lightweight" refers to incremental transport dependencies; ADK's own dependency tree remains. I see it as an optional direct alternative alongside the existing integrations.

Try v0.1.1: Python 3.11+; tested with ADK 2.13.0 and google-genai 2.33.0. There is no PyPI release. In a fresh virtual environment, install the tagged source:

python -m pip install "git+https://github.com/AubinGil/adk-local-models.git@v0.1.1"

Minimal example, assuming an already configured vLLM server; replace your-model with its served model name. The API root includes /v1:

import asyncio
from adk_local_models import OpenAICompatibleLlm
from google.adk.models.llm_request import LlmRequest
from google.genai import types

llm = OpenAICompatibleLlm(
    profile="vllm", base_url="http://localhost:8000/v1", model="your-model"
)
request = LlmRequest(contents=[
    types.Content(role="user", parts=[types.Part(text="Hello")])
])

async def main():
    async for reply in llm.generate_content_async(request):
        print(reply)

asyncio.run(main())

The README has Ollama and llama-server examples and knob prerequisites; the feature comparison links the evidence.

I'd welcome feedback on the response contracts and compatibility reports from others using local models with ADK!