Skip to content

Add inference pool content routing guide#588

Open
danehans wants to merge 3 commits into
agentgateway:mainfrom
danehans:agentgateway-content-routing-inference-pools
Open

Add inference pool content routing guide#588
danehans wants to merge 3 commits into
agentgateway:mainfrom
danehans:agentgateway-content-routing-inference-pools

Conversation

@danehans

@danehans danehans commented Jun 11, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Update the inference quickstart to use llm-d-inference-sim.
  • Add a multiple InferencePools guide that uses AgentgatewayPolicy content routing without an external BBR payload processor.
  • Link the new guide from related inference and content-routing docs.

Testing

  • git diff --check
  • Validated on a fresh kind cluster with GAIE/EPP v1.5.0, agentgateway, Qwen3 and DeepSeek simulator InferencePools.
  • Verified /v1/chat/completions returned 200 for Qwen/Qwen3-32B, deepseek/DeepSeek-r1, and movie-critique.

Signed-off-by: Daneyon Hansen <daneyon.hansen@solo.io>
@danehans danehans force-pushed the agentgateway-content-routing-inference-pools branch from 090f0ed to 5ef04d2 Compare June 11, 2026 00:39
Comment thread content/docs/kubernetes/main/llm/multiple-inference-pools.md Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants