From 9c8592d39cebbe404b3e64243456d179d9b87c4d Mon Sep 17 00:00:00 2001 From: gabriel-farache Date: Mon, 21 Sep 2026 09:40:05 +0200 Subject: [PATCH] docs: document environment agent authentication (inbound + outbound) Adds a dedicated Environment Agent Authentication guide covering the two independent auth surfaces introduced by environment-agent#35 and environment-agent#38: - Inbound: AGENT_AUTH_DISABLED / AGENT_AUTH_ISSUER_URL / AGENT_AUTH_JWT_AUDIENCE protecting the agent's own REST API (external SP registration, provider listing), health-path bypass, RFC 7807 error format, and the intentional audience fail-open behavior. - Outbound: DCM_AUTH_TOKEN / DCM_AUTH_TOKEN_ENDPOINT / DCM_AUTH_CLIENT_ID / DCM_AUTH_CLIENT_SECRET for the agent's own registration/heartbeat calls to the control plane, mode precedence, token refresh and failure behavior, Keycloak service-account setup, and the known reference-realm audience-mapper gap. Addresses the open documentation request from environment-agent#35 (review thread on internal/config/config.go, https://github.com/dcm-project/environment-agent/pull/35#discussion_r4042755293): 'put agent auth enablement on the website... same idea as the CP guide.' Content validated against the latest reviewed commits on both PR branches (config.go, jwt.go, middleware.go, token.go, client.go, main.go, openapi.yaml, README.md, decisions.md) rather than assumed from the PR descriptions alone. Updates the control-plane Authentication guide's Service-providers callout and troubleshooting row to link to the new page instead of a vague 'still landing' note, and cross-links from Local Setup and the Getting Started index. Signed-off-by: Gabriel Farache Co-authored-by: Cursor Signed-off-by: gabriel-farache --- content/docs/getting-started/_index.md | 2 + .../docs/getting-started/authentication.md | 33 ++- .../environment-agent-authentication.md | 208 ++++++++++++++++++ content/docs/getting-started/local-setup.md | 5 + 4 files changed, 231 insertions(+), 17 deletions(-) create mode 100644 content/docs/getting-started/environment-agent-authentication.md diff --git a/content/docs/getting-started/_index.md b/content/docs/getting-started/_index.md index d0ec069..e5f842b 100644 --- a/content/docs/getting-started/_index.md +++ b/content/docs/getting-started/_index.md @@ -22,5 +22,7 @@ Get up and running with DCM. second service provider and create a random selection policy. - **[Authentication](authentication/)** — Enable auth, log in with the CLI or UI, and call the API with bearer tokens. +- **[Environment Agent Authentication](environment-agent-authentication/)** — + Protect the agent's API and authenticate its requests to the control plane. - **[Troubleshooting](troubleshooting/)** — Diagnose issues using container logs and status checks. diff --git a/content/docs/getting-started/authentication.md b/content/docs/getting-started/authentication.md index 919ddd5..fc49692 100644 --- a/content/docs/getting-started/authentication.md +++ b/content/docs/getting-started/authentication.md @@ -26,15 +26,14 @@ The `/api/v1alpha1/health` endpoint stays **unauthenticated** whether or not auth is enabled. > **Service providers:** Service providers reach the control plane through the -> environment agent, not direct HTTP calls. End-to-end authentication for the -> agent and service providers is still in progress (for example -> [environment-agent#38](https://github.com/dcm-project/environment-agent/pull/38) -> and -> [environment-agent#35](https://github.com/dcm-project/environment-agent/pull/35)). +> environment agent, not direct HTTP calls. The agent has its own inbound +> (protecting its API) and outbound (registering with the control plane) +> authentication, documented separately in +> [Environment Agent Authentication](../environment-agent-authentication/). > Enabling auth on the control plane can still affect SP registration and -> instance workflows until that chain is complete. Use auth for CLI, UI, and -> direct API access first, or keep auth disabled while exercising full SP flows -> locally. +> instance workflows if the agent's outbound authentication is not configured to +> match. Use auth for CLI, UI, and direct API access first, or keep auth +> disabled while exercising full SP flows locally. ## How authentication works @@ -226,15 +225,15 @@ returns `403 Forbidden`. ## Troubleshooting -| Symptom | Things to check | -| ------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------- | -| `401 Unauthorized` on API or CLI | Auth enabled; token present and not expired; issuer matches `AUTH_ISSUER_URL`; if `AUTH_JWT_AUDIENCE` is set, token `aud` must match. | -| `dcm login` cannot reach issuer | Host maps `keycloak` (see [host access](#local-compose-host-access-to-keycloak)); issuer matches discovery; TLS if using HTTPS. | -| Issuer / JWKS errors in control-plane logs | `AUTH_ISSUER_URL` reachable from the control-plane container; Keycloak healthy (`auth` profile running). | -| CLI works but UI fails (or reverse) | Backstage SSO and control plane must trust the same IdP and audience; plugin backend URL points at the control plane. | -| `403 Forbidden` with valid token | Actor suspended or deactivated; wait up to `AUTH_CACHE_TTL` after status changes. | -| Service provider errors after enabling auth | SP traffic uses the environment agent; auth for agent and SP paths is still landing. See SP callout in [Overview](#overview). | -| Device flow times out | Complete browser approval within the time shown; retry `dcm login`. | +| Symptom | Things to check | +| ------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `401 Unauthorized` on API or CLI | Auth enabled; token present and not expired; issuer matches `AUTH_ISSUER_URL`; if `AUTH_JWT_AUDIENCE` is set, token `aud` must match. | +| `dcm login` cannot reach issuer | Host maps `keycloak` (see [host access](#local-compose-host-access-to-keycloak)); issuer matches discovery; TLS if using HTTPS. | +| Issuer / JWKS errors in control-plane logs | `AUTH_ISSUER_URL` reachable from the control-plane container; Keycloak healthy (`auth` profile running). | +| CLI works but UI fails (or reverse) | Backstage SSO and control plane must trust the same IdP and audience; plugin backend URL points at the control plane. | +| `403 Forbidden` with valid token | Actor suspended or deactivated; wait up to `AUTH_CACHE_TTL` after status changes. | +| Service provider errors after enabling auth | SP traffic uses the environment agent, which has its own inbound/outbound auth. See [Environment Agent Authentication](../environment-agent-authentication/) and the SP callout in [Overview](#overview). | +| Device flow times out | Complete browser approval within the time shown; retry `dcm login`. | Verify Keycloak readiness when using compose: diff --git a/content/docs/getting-started/environment-agent-authentication.md b/content/docs/getting-started/environment-agent-authentication.md new file mode 100644 index 0000000..acb3bbc --- /dev/null +++ b/content/docs/getting-started/environment-agent-authentication.md @@ -0,0 +1,208 @@ +--- +title: Environment Agent Authentication +type: docs +weight: 8 +--- + +The [environment agent](https://github.com/dcm-project/environment-agent) runs +in a target environment and acts as the intermediary between the DCM control +plane and the Service Providers (SPs) deployed there. It has **two independent** +authentication surfaces, separate from the control-plane authentication covered +in [Authentication](../authentication/): + +1. **Inbound** — protecting the agent's own REST API (external SP registration, + provider listing) from unauthenticated callers. +2. **Outbound** — authenticating the agent's own requests to the control plane + (agent registration and heartbeat) when the control plane has authentication + enabled. + +Both use the same mechanism as the control plane: OIDC-issued JWT bearer tokens, +validated with [Keycloak](https://www.keycloak.org/) in the reference stack. You +can enable either one independently of the other. + +```mermaid +graph LR + SP["External Service Provider"] -->|"Bearer JWT (inbound)"| Agent["Environment Agent"] + Agent -->|"Bearer JWT (outbound)"| CP["Control Plane"] +``` + +## Inbound: protecting the agent's API + +The agent's REST API — external SP registration (`POST /api/v1alpha1/providers`) +and provider listing (`GET /api/v1alpha1/providers`, +`GET /api/v1alpha1/providers/{provider_id}`) — can require a valid JWT bearer +token on every request. This closes the gap called out in the +[environment agent enhancement](https://github.com/dcm-project/enhancements/blob/main/enhancements/environment-agent/environment-agent.md#authentication-and-authorization-for-sp-registration-consolidation): +external SP registration was previously unauthenticated, with network isolation +as the only interim mitigation. + +`GET /api/v1alpha1/health` is the only endpoint that stays public — the bypass +checks both the method and the path, so `GET /health` is unauthenticated but any +other method at that path still requires a token. + +### How it works + +1. The agent discovers the OIDC issuer's JWKS keys at startup (same discovery + flow as the control plane). +2. On each protected request, the agent extracts the + `Authorization: Bearer ` header (case-insensitive scheme) and + validates signature, expiry, issuer, and — if configured — audience. +3. On success, the token's `sub` and `preferred_username` claims are attached to + the request's audit log line. There is no actor store or suspension concept + on the agent side (unlike the control plane): a valid token is sufficient to + call the API. +4. On failure, the agent returns `401 Unauthorized` as an RFC 7807 problem + response with a `WWW-Authenticate: Bearer` header. The response body never + contains the underlying validator error (for example "token expired" or + "signature invalid") — only a fixed `invalid Bearer token` detail. The + specific reason is logged server-side only. + +### Configuration + +| Variable | Purpose | +| ------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `AGENT_AUTH_DISABLED` | When `true` (the default), auth middleware is bypassed. The agent still logs a startup `WARN` ("authentication is disabled") so the insecure default is visible in logs. | +| `AGENT_AUTH_ISSUER_URL` | OIDC issuer URL for JWT validation (for example `http://keycloak:8080/realms/dcm`). Required when `AGENT_AUTH_DISABLED=false`; startup fails otherwise. | +| `AGENT_AUTH_JWT_AUDIENCE` | Expected `aud` claim. Empty (the default) **disables audience validation** — any token from the issuer is accepted regardless of `aud`. The agent logs a startup `WARN` if an issuer is set but the audience is empty. | + +This mirrors the control plane's `AUTH_DISABLED` / `AUTH_ISSUER_URL` / +`AUTH_JWT_AUDIENCE` variables (see +[Control-plane configuration](../authentication/#control-plane-configuration)), +but with an `AGENT_` prefix and its own env vars — the two services are +configured independently, even when pointed at the same Keycloak realm. + +> **Audience is intentionally permissive by default.** Reference Keycloak +> clients for service accounts (used by SPs registering with the agent) don't +> ship with an audience mapper out of the box. Rather than reject every token +> from an unmapped client, the agent skips the audience check when +> `AGENT_AUTH_JWT_AUDIENCE` is empty, and only enforces `aud` matching when you +> explicitly set it. Set it once your SP clients have a matching audience +> mapper. + +### Calling the agent API with a bearer token + +```bash +curl -s \ + -H "Authorization: Bearer ${SP_TOKEN}" \ + -H "Content-Type: application/json" \ + -d '{"name":"db-provider","endpoint":"https://sp.example.com:8080","service_type":"database","schema_version":"v1alpha1"}' \ + http://localhost:8081/api/v1alpha1/providers +``` + +Without a valid token (auth enabled), this returns `401 Unauthorized`: + +```json +{ + "type": "UNAUTHORIZED", + "title": "Unauthorized", + "status": 401, + "detail": "invalid Bearer token", + "instance": "/api/v1alpha1/providers" +} +``` + +`GET /api/v1alpha1/health` always succeeds without a token: + +```bash +curl -s http://localhost:8081/api/v1alpha1/health +``` + +## Outbound: agent-to-control-plane authentication + +When the control plane has authentication enabled (see +[Authentication](../authentication/)), the agent's own registration +(`POST /api/v1alpha1/agents`) and heartbeat +(`PUT /api/v1alpha1/agents/{agent_id}/heartbeat`) calls must carry a valid +bearer token too. Three modes are supported, resolved at startup: + +1. **OAuth2 client credentials** (recommended for production) — the agent + fetches short-lived JWTs from an OIDC token endpoint and refreshes them + automatically before they expire. +2. **Static token** (dev / simple deployments) — a single pre-obtained JWT is + sent as-is on every request, with no refresh. +3. **No auth** (default, backward-compatible) — neither is configured, so no + `Authorization` header is sent. This is required for the unauthenticated + local-setup flow described in + [Local Setup](../local-setup/#verifying-the-deployment). + +If more than one mode is configured, **client credentials takes precedence over +the static token**. + +### Configuration + +| Variable | Mode | Purpose | +| ------------------------- | ------------------ | -------------------------------------------------------------------------------------------------- | +| `DCM_AUTH_TOKEN_ENDPOINT` | Client credentials | OIDC token endpoint (for example `http://keycloak:8080/realms/dcm/protocol/openid-connect/token`). | +| `DCM_AUTH_CLIENT_ID` | Client credentials | OAuth2 client ID for the agent's service account. | +| `DCM_AUTH_CLIENT_SECRET` | Client credentials | OAuth2 client secret. | +| `DCM_AUTH_TOKEN` | Static token | A pre-obtained JWT sent unchanged on every request. | + +All three client-credentials variables must be set together — a partial +configuration (for example an endpoint without a client ID) is a **startup-fatal +error** that names the missing variables. `DCM_AUTH_TOKEN_ENDPOINT` must also be +an absolute, host-qualified `http(s)://` URL. + +> **Caution:** `http://` token endpoints send the client secret unencrypted on +> every token request. The agent logs a startup warning in this case; prefer +> `https://` outside local development. The agent also refuses to follow any +> redirect returned by the token endpoint, so the secret is never resent to a +> different origin. + +### Token handling + +- Client-credentials tokens are cached in memory and refreshed proactively — 10 + seconds before the `expires_in` deadline from the token response, not + reactively on a `401`. +- A token fetch failure does **not** crash the agent: the affected registration + or heartbeat call fails and is retried by the agent's existing backoff logic + (`DCM_REGISTRATION_INITIAL_BACKOFF` / `DCM_REGISTRATION_MAX_BACKOFF`), exactly + as a network failure would be. +- **Static tokens do not refresh.** Once the token expires, the control plane + returns `401`, which the agent treats as non-retryable — registration halts + until the agent is restarted with a fresh token. Use static tokens only where + the token's lifetime safely exceeds the agent's expected uptime. + +### Keycloak setup + +1. Create a service-account client in the DCM realm (for example + `environment-agent`). +2. Enable **Client authentication** and **Service accounts roles**. +3. Assign the role(s) the control plane expects for agent registration. +4. Attach an audience mapper for `aud=dcm-api` to the client (see the known gap + below). +5. Set `DCM_AUTH_TOKEN_ENDPOINT`, `DCM_AUTH_CLIENT_ID`, and + `DCM_AUTH_CLIENT_SECRET` to that client's token endpoint and credentials. + +> **Known gap — reference realm has no audience mapper for the agent's client.** +> The control plane's reference realm ships a `dcm-api` audience mapper on the +> `dcm-proxy` and `dcm-cli` clients only. A service-account client you create +> for the agent won't have it by default, so its tokens won't carry +> `aud=dcm-api`. If the control plane has `AUTH_JWT_AUDIENCE=dcm-api` set (as +> the reference `.env.example` does when auth is enabled), it will reject the +> agent's registration and heartbeat requests with `401` until you attach an +> equivalent audience mapper to the agent's client by hand. + +### Kubernetes secrets + +Store the client secret in a Kubernetes `Secret` and inject it with `envFrom` +rather than embedding it in a pod spec or `ConfigMap`: + +```yaml +envFrom: + - secretRef: + name: environment-agent-auth +``` + +## Troubleshooting + +| Symptom | Things to check | +| -------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| SP registration returns `401` from the agent | Auth enabled on the agent (`AGENT_AUTH_DISABLED=false`); the SP's token is valid and not expired; issuer matches `AGENT_AUTH_ISSUER_URL`; if `AGENT_AUTH_JWT_AUDIENCE` is set, the token's `aud` must match. | +| Agent registration/heartbeat to the control plane returns `401` | Control plane has `AUTH_JWT_AUDIENCE` set but the agent's client has no matching audience mapper (see the known gap above); confirm with the control-plane logs which claim failed. | +| Agent fails to start with `AGENT_AUTH_ISSUER_URL: required...` | `AGENT_AUTH_DISABLED=false` but no issuer was set — set `AGENT_AUTH_ISSUER_URL` or leave auth disabled. | +| Agent fails to start with a "partial client-credentials auth config" error | Only some of `DCM_AUTH_TOKEN_ENDPOINT` / `DCM_AUTH_CLIENT_ID` / `DCM_AUTH_CLIENT_SECRET` are set — set all three or none. | +| Registration halts permanently after working for a while | Static `DCM_AUTH_TOKEN` expired (no refresh) — switch to client credentials, or restart the agent with a fresh token. | +| `AGENT_AUTH_JWT_AUDIENCE` startup warning | Expected when an issuer is set without an audience — informational unless you intended to enforce `aud`. | + +For control-plane-side issuer, JWKS, and Keycloak troubleshooting, see +[Authentication troubleshooting](../authentication/#troubleshooting). diff --git a/content/docs/getting-started/local-setup.md b/content/docs/getting-started/local-setup.md index 532a412..ae81610 100644 --- a/content/docs/getting-started/local-setup.md +++ b/content/docs/getting-started/local-setup.md @@ -70,6 +70,11 @@ curl http://localhost:8080/api/v1alpha1/agents When authentication is enabled, add `Authorization: Bearer ` to API calls. See [Authentication](authentication/#authenticating-api-requests). +> **Note:** This is the control plane's own auth for the `/agents` endpoint. The +> environment agent process itself has separate, independent authentication for +> its own API and for its calls back to the control plane — see +> [Environment Agent Authentication](environment-agent-authentication/). + ## Setting Up the CLI The DCM CLI (`dcm`) lets you interact with the DCM control plane from the