feat(omnigent): managed hosts and automatic Gateway setup

Makefile

@@ -79,12 +79,12 @@ help: ## Show this help
 
 # ── Workflows ────────────────────────────────────────
 
-deploy: create-app grant-omnigent-host sync deploy-app ## Full deploy (create app, grant Omnigent host IAM, sync, deploy)
+deploy: create-app configure-gateway-resources grant-omnigent-host sync deploy-app ## Full deploy (create app, attach Gateway models, grant Omnigent IAM, sync, deploy)
 	@echo ""
 	@echo "Deployment complete! App URL:"
 	@databricks apps get $(APP_NAME) --profile $(PROFILE) --output json 2>/dev/null | python3 -c "import sys,json; print(json.load(sys.stdin).get('url','(pending)'))"
 
-redeploy: grant-omnigent-host sync deploy-app ## Redeploy: (re)grant Omnigent host IAM + sync + deploy
+redeploy: configure-gateway-resources grant-omnigent-host sync deploy-app ## Redeploy: refresh Gateway models + Omnigent IAM, sync, deploy
 	@echo ""
 	@echo "Redeployment complete!"
 
@@ -206,7 +206,7 @@ configure-git-credential: ## Add a Git credential to the app SP for private repo
 		&& echo "    Git credential added to SP $$sp_id for '$(APP_NAME)'." \
 		|| echo "    git-credentials create failed (credential may already exist for $(GIT_PROVIDER))."
 
-deploy-git: ## Deploy the app from the configured Git ref ($(GIT_REF_TYPE)=$(GIT_REF))
+deploy-git: configure-gateway-resources ## Attach Gateway models, then deploy from configured Git ref ($(GIT_REF_TYPE)=$(GIT_REF))
 	@echo "==> Deploying '$(APP_NAME)' from Git $(GIT_REF_TYPE)='$(GIT_REF)'..."
 	@databricks apps deploy $(APP_NAME) --profile $(PROFILE) --no-wait \
 		--json '{"git_source":{"$(GIT_REF_TYPE)":"$(GIT_REF)"}}'
@@ -217,6 +217,18 @@ redeploy-git: grant-omnigent-host deploy-git ## (Re)grant Omnigent host IAM, the
 	@echo ""
 	@echo "Git redeployment complete!"
 
+# ── AI Gateway model resources ─────────────────────
+
+AUTO_CONFIGURE_GATEWAY ?= true
+
+configure-gateway-resources: ## Grant app SP UC EXECUTE (v3) + CAN_QUERY (legacy) on READY chat models
+	@if [ "$(AUTO_CONFIGURE_GATEWAY)" = "true" ]; then \
+		echo "==> Configuring AI Gateway model resources for '$(APP_NAME)'..."; \
+		python3 configure_gateway_resources.py --profile $(PROFILE) --app $(APP_NAME); \
+	else \
+		echo "==> AI Gateway resource configuration disabled (AUTO_CONFIGURE_GATEWAY=$(AUTO_CONFIGURE_GATEWAY))."; \
+	fi
+
 # ── Omnigent host resources ─────────────────────────
 # The generic app.yaml resolves workspace-specific Omnigent values at runtime
 # via valueFrom resource references. This target attaches those resources to
@@ -226,9 +238,10 @@ redeploy-git: grant-omnigent-host deploy-git ## (Re)grant Omnigent host IAM, the
 OMNIGENT_SERVER_URL ?=
 OMNIGENT_SECRET_SCOPE ?= coda-omnigent
 OMNIGENT_SECRET_KEY ?= omnigent-server-url
+OMNIGENT_CLIENT_ID_SECRET_KEY ?= omnigent-server-client-id
 
-attach-omnigent-resources: ## Attach the per-app omnigent-wheels (UC Volume) + omnigent-server-url (Secret) resources the generic app.yaml resolves via valueFrom
-	@# The generic app.yaml references two resource keys at runtime:
+attach-omnigent-resources: ## Attach the workspace-specific Omnigent volume, URL, and server-SP resources
+	@# The generic app.yaml references three resource keys at runtime:
 	@#   OMNIGENTS_WHEEL_SPEC    valueFrom: omnigent-wheels
 	@#   OMNIGENTS_SERVER_URL    valueFrom: omnigent-server-url
 	@# This target attaches those resources to the app (merging with existing
@@ -247,10 +260,12 @@ attach-omnigent-resources: ## Attach the per-app omnigent-wheels (UC Volume) + o
 	@./attach_omnigent_resources.sh \
 		--profile $(PROFILE) \
 		--coda-app $(APP_NAME) \
+		--server-app $(OMNIGENT_SERVER_APP) \
 		--server-url $(OMNIGENT_SERVER_URL) \
 		--wheel-volume $(WHEEL_VOLUME) \
 		--secret-scope $(OMNIGENT_SECRET_SCOPE) \
-		--secret-key $(OMNIGENT_SECRET_KEY)
+		--secret-key $(OMNIGENT_SECRET_KEY) \
+		--client-id-secret-key $(OMNIGENT_CLIENT_ID_SECRET_KEY)
 
 # ── Monitoring ───────────────────────────────────────
 

README.md

@@ -185,26 +185,26 @@ Tracing setup is skipped gracefully when `APP_OWNER` is not set (e.g., local dev
 
 ## Omnigent Host Integration
 
-CoDA can register itself as a persistent **[Omnigent](https://github.com/omnigent-ai/omnigent) agent host** — an always-on target the Omnigent server can drive coding-agent sessions into. Those sessions run *inside this container* and use the same filesystem as browser terminals. They authenticate to Databricks as the CoDA app service principal, not as the interactive browser user, so their Unity Catalog authority may differ. A deployed CoDA app becomes both an interactive terminal **and** a headless host that survives restarts and redeploys.
+CoDA can act as a managed **[Omnigent](https://github.com/omnigent-ai/omnigent) agent host**. The Omnigent server acquires a fenced lease before connecting the host and launching coding-agent sessions inside the container. Each session receives a separate workspace, and abandoned idle leases are reaped automatically. Runners authenticate to Databricks as the CoDA app service principal, not as the interactive browser user, so their Unity Catalog authority may differ.
 
-**Off by default.** With `OMNIGENTS_SERVER_URL` unset, none of this runs and CoDA behaves exactly as before. This is opt-in, environment-specific wiring — the committed `app.yaml` keeps it commented out.
+**Disabled by default.** Attached resources or `OMNIGENTS_SERVER_URL` alone do not activate the integration. CoDA exposes the managed control plane only when `CODA_OMNIGENT_MODE=managed`; persistent boot-host registration is not supported.
 
 ### Turning it on
 
-Set three variables in your deployed `app.yaml` (see `app.yaml.workshop` for a ready-to-copy overlay template):
+Set managed mode and configure the server identity and host artifacts through app resources (the `attach-omnigent-resources` target attaches all three):
 
 ```yaml
 # app.yaml
 env:
-  # The Omnigent server this app registers against on boot.
+  - name: CODA_OMNIGENT_MODE
+    value: "managed"
+  # Client ID of the only Omnigent server SP allowed to control this host.
+  - name: OMNIGENT_SERVER_SP_CLIENT_ID
+    valueFrom: omnigent-server-client-id
   - name: OMNIGENTS_SERVER_URL
-    value: "https://<your-omnigent-app>.<region>.databricksapps.com"
-  # UC Volume holding the omnigent host wheels (app SP needs READ_VOLUME).
+    valueFrom: omnigent-server-url
   - name: OMNIGENTS_WHEEL_SPEC
-    value: "/Volumes/<catalog>/<schema>/artifacts/wheels"
-  # Optional: force-reinstall the host CLI on boot while rolling out a new wheel.
-  - name: OMNIGENTS_FORCE_REINSTALL
-    value: "1"
+    valueFrom: omnigent-wheels
 ```
 
 Before deploying, grant the CoDA app service principal `CAN_USE` on the
@@ -216,7 +216,7 @@ the complete prerequisite set:
 make grant-omnigent-host PROFILE=<profile> APP_NAME=<coda-app>
 ```
 
-On boot, `initialize_app()` calls `start_host()`, which — only when `OMNIGENTS_SERVER_URL` is set — installs the `omnigents host` CLI from the wheel volume and launches it as a supervised background process that dials the server over an outbound WSS tunnel.
+On boot, CoDA captures its service-principal credentials and waits in `idle`. The authorised Omnigent server acquires a lease and then asks CoDA to launch the supervised outbound WSS host tunnel. A stale lease generation cannot connect, disconnect, or scrub a newer allocation.
 
 ### Two credentials, two jobs
 
@@ -238,15 +238,17 @@ The non-obvious part of this design is that the host uses **two separate credent
 
 ### Runtime controls
 
-Beyond boot registration, the host can be driven at runtime:
+The managed endpoints require the configured Omnigent server SP and return `404` while managed mode is disabled:
 
 | Endpoint | Method | Purpose |
 |----------|--------|---------|
-| `/api/omnigents-status` | GET | Host-integration state (FR-9 observability) |
-| `/api/omnigent-host/status` | GET | Current runtime host state |
-| `/api/omnigent-host/connect` | POST | Start a host tunnel for a supplied `server_url` |
-| `/api/omnigent-host/disconnect` | POST | Stop the active host tunnel |
-| `/api/omnigent-host/share` | POST | Share the SP-owned host with a connecting user |
+| `/api/omnigents-status` | GET | Sanitised browser-visible integration state |
+| `/api/omnigent-host/status` | GET | Current managed-host state |
+| `/api/omnigent-host/lease` | POST | Acquire or adopt the fenced user lease |
+| `/api/omnigent-host/workspaces` | POST | Allocate a session-specific workspace |
+| `/api/omnigent-host/connect` | POST | Connect the leased host tunnel |
+| `/api/omnigent-host/disconnect` | POST | Release and optionally scrub the matching lease |
+| `/api/omnigent-host/runner-log/<session-id>` | GET | Return a bounded runner diagnostic tail |
 
 ### Related
 
@@ -261,10 +263,10 @@ Beyond boot registration, the host can be driven at runtime:
 1. Click [**Use this template**](https://github.com/datasciencemonkey/coding-agents-databricks-apps/generate) to create your own repo
 2. Go to **Databricks → Apps → Create App**
 3. Choose **Custom App** and connect your new repo
-4. Deploy
-5. Open the app — paste a short-lived PAT when prompted on first terminal session
+4. From your checkout, run `make configure-gateway-resources PROFILE=<profile> APP_NAME=<app>`
+5. Deploy, then open the app and paste a short-lived PAT if prompted
 
-That's it. No secrets to configure, no pre-deployment setup.
+> **Required:** a plain Apps UI deployment cannot grant Unity AI Gateway v3 model-service privileges. The configuration target grants the app SP `USE CATALOG`, `USE SCHEMA`, and least-privilege `EXECUTE`; without it Pi and Claude report `404 '<model>' does not exist`. The repository's normal `make deploy`, `make redeploy`, and `make deploy-git` workflows run it automatically.
 
 [→ Full deployment guide](docs/deployment.md) — environment variables, gateway config, and advanced options.