Repository navigation
Add an agentgateway community blueprint for GKE - #6334
Conversation
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request introduces a new community blueprint for deploying a private, multi-provider generative AI gateway on GKE. It streamlines the setup of regional GKE clusters with integrated load balancing and secure identity management, while providing robust validation mechanisms to ensure consistent deployment states and model connectivity. Highlights
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here. Footnotes
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces a new community blueprint for deploying a multi-provider agentgateway on GKE, along with supporting Terraform modules (agentgateway-identity and vertex-ai-prediction), integration tests, and validation scripts. The code review identified several critical issues and improvement opportunities: potential AttributeError crashes in readiness.py due to unhandled nullable status fields, an invalid sampleRate value in the GCPBackendPolicy template, compatibility issues with domain-scoped projects in the Workload Identity configuration, a potential StopIteration crash in run-in-cluster.py when no running proxy pod is found, and a recommendation to catch TimeoutError directly in smoke-test.py for more robust network retries.
815338c to
bafd2ff
Compare
AdarshK15
added
the
release-key-new-features
bafd2ff to
c772375
Compare
|
@Neelabh94 when you have a moment, PTAL and kickoff CI. |
|
this PR has been inactive for 2 days and has no unresolved comments. @arpit974, @saara-tyagi27, please review. |
|
@danehans, There is a merge conflict, could you please rebase the PR. |
c772375 to
fa01663
Compare
|
@AdarshK15 I rebased and resolved the merge conflict. Please re-kick CI. |
fa01663 to
41be1dc
Compare
|
this PR has been inactive for 2 days and has no unresolved comments. @arpit974, @saara-tyagi27, please review. |
|
This is a second reminder that this PR has been inactive for 4 days and has no unresolved comments. @arpit974, @saara-tyagi27, please review. |
|
@arpit974 @saara-tyagi27 do you need anything from me to get the review process started with this PR? |
|
/gcbrun |
|
this PR has been inactive for 2 days and has no unresolved comments. @arpit974, @saara-tyagi27, please review. |
|
This is a second reminder that this PR has been inactive for 4 days and has no unresolved comments. @arpit974, @saara-tyagi27, please review. |
|
@arpit974 wanted to follow-up on this PR. Is there anything I can do to help move the review along? |
|
this PR has been inactive for 2 days and has no unresolved comments. @arpit974, @saara-tyagi27, please review. |
7b6d586 to
ad44e78
Compare
|
Pushed 848f577 with the rendering and smoke-retry fixes. Full Linux The live run used #6536 and existing identities. OpenAI/Anthropic calls were skipped as requested. Cloud Build checks need a collaborator to run Cleanup is complete. The test cluster and associated resources are gone; the administrator-provisioned accounts and prediction role remain. |
|
/gcbrun |
Neelabh94
left a comment
Neelabh94
left a comment
There was a problem hiding this comment.
Thanks @danehans for addressing all 16 items from the previous round and validating the updated blueprint on a live GKE cluster! The PR is largely LGTM — I left a few small follow-up notes on the existing threads where the latest changes introduced minor edge cases, along with two small nits in run_in_cluster.py. Once those are tidied up, this should be ready to approve.
|
@aslam-quad can you |
|
/gcbrun |
There was a problem hiding this comment.
Thanks for addressing all the follow-up items and adding unit tests for the edge cases (and for verifying the v1.7.0 upgrade on GKE 1.35)! Everything looks clean and well-tested — LGTM.
cc @arpit974 @saara-tyagi27 to take a look and provide the second approval if everything looks good to them too.
|
Hi @danehans Could you please take a look at the merge conflicts. Thanks |
Deploy a private multi-provider gateway with Workload Identity and optional administrator-provisioned service accounts. Add standalone Gateway API support, readiness checks, and automated integration validation. Keep monitoring configuration stable on reapply. Signed-off-by: Daneyon Hansen <daneyon.hansen@solo.io>
Recognize missing-secret diagnostics in v1.6.0 without treating a partially valid backend as ready for requests. Signed-off-by: Daneyon Hansen <daneyon.hansen@solo.io>
Signed-off-by: Daneyon Hansen <daneyon.hansen@solo.io>
Run readiness hooks with configuration updates and reject partial backends. Simplify IAM wiring, colocate smoke tools, and strengthen failure coverage. Signed-off-by: Daneyon Hansen <daneyon.hansen@solo.io>
Render readiness manifests before passing them into stage templates. Allow bounded retries while load-balancer backends propagate, and record Linux and live deployment/update validation. Signed-off-by: Daneyon Hansen <daneyon.hansen@solo.io>
Share readiness checks, reject unsuitable proxy pods, and wait for synthetic backend cleanup. Add focused regression tests and clearer static validation diagnostics. Signed-off-by: Daneyon Hansen <daneyon.hansen@solo.io>
Update the shared chart version and catalog, and record Linux and live upgrade validation results. Signed-off-by: Daneyon Hansen <daneyon.hansen@solo.io>
Omit unset provider Secret arguments and test all provider combinations through Ansible. Refresh generated indexes after rebasing on develop. Signed-off-by: Daneyon Hansen <daneyon.hansen@solo.io>
95beeb3 to
0254b4b
Compare
|
/gcbrun |
6e89b0b
into
GoogleCloudPlatform:develop
|
@Neelabh94 and all thanks a bunch for your support getting this PR through the finish line! |
Adds a regional, CPU-only GKE blueprint using agentgateway v1.7.0 for a private multi-provider gateway. Gemini and OpenAI expose Chat Completions; Anthropic exposes native
/v1/messages. Implements phase 1 of #6213 using shared IAM modules, with documented support for administrator-provisioned identities.Gateway API can be enabled independently of Inference Extension. Each gateway/ingress release includes its readiness hook, which runs on installation and configuration changes. Readiness rejects partially valid backends and checks ingress policy attachment. The example includes a companion deployment file, smoke tests, schema checks and a manual integration playbook; upstream scheduling remains separate work. Node-pool lifecycle documentation is split into #6535.
This revision consolidates Helm releases and changes IAM resource addresses. Recreate disposable deployments from earlier PR revisions; existing deployments need migration planning. The monitoring-order workaround is removed following the upstream Google provider 8.x upgrade.
November 2 rebase and provider-argument fix
Rebased onto
developat665778f29, preserving upstream catalog and configuration-discovery changes. Optional provider Secret flags are now omitted when unset. An Ansible regression check covers OpenAI only, Anthropic only, both and neither; it runs throughmake test-agentgatewaywithout adding Ansible to the isolated pre-commit pytest environment.make test-agentgateway(Go regression tests and 60 Python tests), Ansible syntax, pinned-chart static validation, and actual Ansible-to-gcluster createchecks for all three enabled-provider combinations. These creation checks used offline cloud-validator settings and created no cloud resources._DaemonThreadPoolExecutor._initializer). Broad Go tests also hit unchanged upstream config/cache and shell-signal test failures on macOS. These files matchorigin/developand were not modified.Validation
Validated on October 28, 2026:
make testsandmake agentgateway-staticpassed, including all 59 Python helper tests and optional-provider schema checks against both v1.7.0 charts. Applicable pre-commit checks passed.gclusterto v1.7.0. Both Helm releases and the controller/proxy images were verified at v1.7.0. The test used GKE1.35.8-gke.1919000, Gateway API v1.5.0, Google provider 8.6.0 and administrator-provisioned identities.gemini-2.5-flashand the synthetic provider passed synchronous, streaming and metrics checks through the internal load balancer. Synthetic resources and server pods were absent after runner exit, including immediately before deployment teardown.PartiallyValid; adding a dummy Secret completed the update. After restoring the base configuration, unchanged reapplication reported all four groups already applied.Earlier validation and review regression coverage are recorded in
community/examples/agentgateway-gke/TESTING.md. Cloud Build requires a collaborator's/gcbrunbefore its checks can execute.Submission Checklist
NOTE: Community submissions can take up to 2 weeks to be reviewed.
Please take the following actions before submitting this pull request.