feat(accounts): a local runtime may list loopback stubs a callable source can reach

MODULE.md

@@ -299,8 +299,9 @@ Two client sources, which cannot collide — a registry slug can never spell
   fetched (bounded size and time, **no redirect at all** per CIMD draft-02 §5.1,
   and guarded at the dial against every non-public destination — the IANA
   special-purpose registries for both families, not just loopback and RFC 1918,
-  with IPv6 bounded to the `2000::/3` global unicast allocation; every other
-  tenant-directed egress uses the same classifier, listed in
+  with IPv6 bounded to the `2000::/3` global unicast allocation; the other
+  guarded egress paths use the same classifier, and the paths it covers and the
+  OIDC clients it does not are listed in
   [CALLABLE_SOURCES.md](./module/CALLABLE_SOURCES.md#oauth-and-disclosure)),
   validated, and used as a public client for that one flow. Failures and invalid
   documents are **not** cached (§5.2), so a publisher who fixes one is not made

module/CALLABLE_SOURCES.md

@@ -139,11 +139,13 @@ Transport uses the resolved-IP egress guard and same-origin redirect policy.
 The guard runs at connect time, on the address DNS returned, for the operation
 call and every OAuth token exchange alike, and admits only an address on the
 public internet. That judgement is `auth.IsPublicDestination`, the one
-classifier for every tenant-directed egress in this host: callable sources
-(operation calls and OAuth token exchanges), the documentation crawler, the
-object-store connector, outbound webhooks and the Slack notifier that shares
-their client, and Client ID Metadata Document fetches. Each of those guards is
-tested against the same vectors, `pkg/auth/egresstest`. It reads the
+classifier shared by these guarded egress paths: callable sources (operation
+calls and OAuth token exchanges), the documentation crawler, the object-store
+connector, outbound webhooks and the Slack notifier that shares their client,
+and Client ID Metadata Document fetches. Each of those guards is tested against
+the same vectors, `pkg/auth/egresstest`. The per-organization OIDC discovery,
+JWKS and token clients (`pkg/auth/oidc`) are not guarded. No RPC exposes
+`ConfigureOrgIdentityProvider` today, so no tenant can name their destination. It reads the
 IANA IPv4 and IPv6 special-purpose registries rather than Go's `net.IP`
 predicates, which call carrier NAT (`100.64.0.0/10`), benchmarking, reserved
 and the IPv6 transition prefixes "global unicast", and it bounds IPv6 to
@@ -152,12 +154,75 @@ Teredo are refused outright rather than decoded, because a network-specific
 NAT64 prefix cannot be recognised and a decoded public address is still reached
 through a translator on the host's own network. A v4-mapped `::ffff:a.b.c.d` is
 judged as `a.b.c.d`. A zoned address, or one the guard cannot parse, is refused.
+The one exception is a local runtime's explicit list of loopback addresses,
+described in [Local egress](#local-egress); every address it does not list is
+still judged by the guard.
 Named refusal/receipt/audit constructors discard provider error bodies and raw
 transport errors. Credential reflection in a successful provider body is refused.
 Audit records actor, source, operation, effect, effect ID and the closed outcome
 vocabulary (`committed`, `refused`, `unresolved`, `rate_limited`), never input,
 output, tokens or URLs. Failed operation resolution uses an `UNKNOWN` effect.
 
+## Local egress
+
+A composition booted locally can prove an OAuth REST source end to end against a
+stub provider and API running beside the host. It lists the stub's exact
+addresses in `datasource-oauth/local_egress`, a configuration value
+([configuration groups](configurations/README.md#datasource-oauth)):
+
+```
+local_egress=127.0.0.1:18080,[::1]:18081
+```
+
+The allowance holds to these rules:
+
+- **Empty by default.** The shipped value is empty, and an empty value changes
+  nothing anywhere.
+- **Refused outside a local runtime.** A non-empty value refuses startup, naming
+  the key, before any other startup check runs. A local runtime has to satisfy two
+  facts the host already reads. First, the Codefly environment is `local`
+  (`codefly.IsLocal`, which every other local-only exemption reads). Second, the
+  process is not in a Kubernetes pod: it has no `KUBERNETES_SERVICE_HOST` and no
+  projected service-account token or CA at their fixed paths. The pod check reads
+  the same facts as the solution host's in-cluster identity. A host started by
+  `codefly run` has none of them. A local Kubernetes cluster is refused too.
+
+  What defeating both checks takes:
+  - **The environment** is one variable, set by whatever renders the deployment.
+  - **The pod facts** come from the kubelet, and no configuration value can set
+    or clear them. A Codefly-rendered service pod mounts no service-account token
+    by default (`automountServiceAccountToken: false`), so in such a pod the only
+    pod fact is `KUBERNETES_SERVICE_HOST`.
+  - So a deployment passes both checks only with control of the pod spec:
+    `CODEFLY__ENVIRONMENT=local` plus an explicit container-env override that
+    clears `KUBERNETES_SERVICE_HOST`. It cannot pass them through a
+    configuration value.
+  - A deployment that can name itself `local` has already lifted the refusals
+    of the dev identity provider and the ephemeral signing key, which are worse
+    than this allowance.
+- **Explicit loopback `ip:port` entries only.** Each entry is one loopback IP
+  literal and one port, written canonically. A hostname, range, CIDR, wildcard,
+  zone, IPv4-mapped spelling, port 0 or non-loopback address refuses the whole
+  value. Loopback never leaves the host's own network namespace. Even a host that
+  got past the runtime check could reach only that namespace: never a metadata
+  service, the cluster network or another tenant. Within that namespace,
+  loopback reaches everything listening there:
+  - on a developer's machine, every local listener, so the developer picks the
+    port;
+  - in a pod, every container in the pod, sidecars included. A mesh proxy's
+    admin port is on loopback.
+- **Never widens the public-address guard.** The allowance is checked at dial
+  time against the resolved address and port, ahead of the guard. Any address it
+  does not list is judged by the guard exactly as before. The allowance lives in
+  its own file (`apisource/local_egress.go`) beside the guard, so each can be
+  reviewed separately.
+- **A value, never a file.** Codefly delivers it like every other group value.
+
+It applies to the token exchange (refresh, `client_credentials`,
+authorization-code) and the operation call alike. A Docker-runtime local run
+reaches a stub through a private address and is not covered. Run the host
+natively, which is the `codefly run` default.
+
 ## Generation and qualification
 
 Core's option import closure is copied byte for byte under the accounts proto's

module/WEBHOOKS.md

@@ -3,11 +3,12 @@
 Webhook endpoints must be public `https://` URLs on port 443. Accounts resolves
 all A/AAAA records when an endpoint is registered and again immediately before
 each connection. If any answer is not on the public internet, the request is
-rejected. That judgement is `auth.IsPublicDestination`, the classifier every
-tenant-directed egress in this host shares (see
-[CALLABLE_SOURCES.md](./CALLABLE_SOURCES.md#oauth-and-disclosure)): the IANA
-IPv4 and IPv6 special-purpose registries, with IPv6 bounded to the `2000::/3`
-global unicast allocation. A connection is pinned to the address that was
+rejected. That judgement is `auth.IsPublicDestination`, the classifier the
+host's guarded egress paths share. The paths it covers, and the OIDC clients it
+does not, are listed in
+[CALLABLE_SOURCES.md](./CALLABLE_SOURCES.md#oauth-and-disclosure). It reads the
+IANA IPv4 and IPv6 special-purpose registries, with IPv6 bounded to the
+`2000::/3` global unicast allocation. A connection is pinned to the address that was
 judged. Redirects are returned as failed attempts and never followed.
 Kubernetes egress policy is an independent second layer over the
 special-purpose ranges it lists; the application check is the complete one.

module/configurations/README.md

@@ -6,15 +6,25 @@ consuming workspaces override a group rather than editing module-owned files.
 
 ## datasource-oauth
 
-Accounts reads three JSON maps. This version supports one deployment app,
-keyed by `api`; other provider keys are refused, so configuration cannot replace
-a connector-owned linker:
+Accounts reads three JSON maps and one local-only list. This version supports
+one deployment app, keyed by `api`; other provider keys are refused, so
+configuration cannot replace a connector-owned linker:
 
 | Key | Delivery | Content |
 | --- | --- | --- |
 | `providers` | value | `authorize_url`, `token_url`, and `scopes` for each provider |
 | `client_id` | value | The deployment's registered client identifier |
 | `client_secret` | secret | The confidential client secret; omit for public clients |
+| `local_egress` | value | Local runtime only: loopback `ip:port` entries a callable source may dial. Empty by default |
+
+`local_egress` lets a host booted on a developer's machine reach a stub OAuth
+provider and API beside it, for example `127.0.0.1:18080,[::1]:18081`. Each
+entry is one loopback IP literal and one port, written canonically: no hostname,
+range, CIDR, wildcard, zone or port 0. It applies to every callable-source dial,
+both the token exchange and the operation call. Any non-empty value refuses
+startup unless the Codefly environment is `local` **and** the process is not in
+a Kubernetes pod. A deployed composition that sets it does not start. See
+[Local egress](../CALLABLE_SOURCES.md#local-egress).
 
 The local provider and client-id maps are empty, which disables generic sign-in.
 The secret map contains only a local placeholder for `api`. For example,

module/configurations/local/datasource-oauth.env

@@ -3,3 +3,8 @@
 providers={}
 # Public app identifiers keyed by the same connector. Empty disables linking.
 client_id={}
+# Local runtime only: loopback ip:port entries a callable source may dial, so a
+# host booted on a developer's machine can reach a stub provider beside it.
+# e.g. 127.0.0.1:18080,[::1]:18081. Empty by default; any value refuses startup
+# outside a local runtime (module/CALLABLE_SOURCES.md, "Local egress").
+local_egress=

module/services/accounts/code/pkg/auth/egresstest/vectors.go

@@ -1,5 +1,5 @@
 // Package egresstest is the set of addresses every egress guard in this host
-// is held to. Each guard that decides whether a connection may leave for a
+// is held to (the paths with a guard are listed in auth/public_address.go). Each guard that decides whether a connection may leave for a
 // tenant-named destination answers through auth.IsPublicDestination, and each
 // guard's own test runs these vectors against it, so a guard that stops using
 // that classifier — or a classifier change that re-admits a range — fails a

module/services/accounts/code/pkg/auth/public_address.go

@@ -14,7 +14,10 @@ import (
 // it: callable sources (datasource/apisource), the crawler and object-store
 // connectors, outbound webhooks and the Slack notifier that shares their client
 // (business.WebhookEndpointPolicy), and Client ID Metadata Document fetches.
-// Each of those guards' tests holds the vectors in auth/egresstest.
+// Each of those guards' tests holds the vectors in auth/egresstest. The
+// per-organization OIDC discovery, JWKS and token clients (auth/oidc) have no
+// guard: nothing exposes ConfigureOrgIdentityProvider over RPC, so no tenant
+// names their destination today.
 //
 // It does not compose Go's net.IP predicates, because they answer a different
 // question. net.IP.IsGlobalUnicast is not a public-routability test — it

module/services/accounts/code/pkg/business/datasource_local_egress_integration_test.go

@@ -0,0 +1,121 @@
+//go:build !pure
+
+package business_test
+
+import (
+	"context"
+	"encoding/json"
+	"net/http"
+	"net/http/httptest"
+	"sync/atomic"
+	"testing"
+	"time"
+
+	"accounts/pkg/auth"
+	"accounts/pkg/business"
+	"accounts/pkg/datasource/apisource"
+	"accounts/pkg/datasource/connector/connectortest"
+	"accounts/pkg/datasource/operations"
+	gen "accounts/pkg/gen/saas/accounts/v1"
+
+	"github.com/stretchr/testify/require"
+	"google.golang.org/protobuf/proto"
+)
+
+// #1087, at the service level: an httptest provider and API beside the host,
+// reached through a client_credentials source with the real store, the real
+// Vault envelope and the REAL guarded transport — neither the token exchange
+// nor the operation client is substituted. The only thing that lets the dials
+// through is a local runtime's allowance naming the stub's exact address.
+func TestClientCredentialsSourceReachesALocalStubThroughTheEgressAllowance(t *testing.T) {
+	clearData(t)
+	actor, org := mustUserAndOrg(t, testCtx, "local-egress@example.com", "local-egress", "Acme")
+	svc, _ := realOAuthService(t)
+	clientSecret := randomCredential(t)
+	const accessToken = "stub-access-token-0123456789abcdef"
+	var tokenCalls, apiCalls atomic.Int32
+	stub := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		switch r.URL.Path {
+		case "/token":
+			tokenCalls.Add(1)
+			if err := r.ParseForm(); err != nil || r.PostForm.Get("grant_type") != "client_credentials" ||
+				r.PostForm.Get("client_id") != "example-client" || r.PostForm.Get("client_secret") != clientSecret {
+				w.WriteHeader(http.StatusUnauthorized)
+				_, _ = w.Write([]byte(`{"error":"invalid_client"}`))
+				return
+			}
+			_ = json.NewEncoder(w).Encode(map[string]any{"access_token": accessToken, "expires_in": 3600})
+		case "/items/a":
+			apiCalls.Add(1)
+			if r.Header.Get("Authorization") != "Bearer "+accessToken {
+				w.WriteHeader(http.StatusUnauthorized)
+				return
+			}
+			_, _ = w.Write([]byte(`{}`))
+		default:
+			http.NotFound(w, r)
+		}
+	}))
+	defer stub.Close()
+	t.Cleanup(func() { _ = apisource.ConfigureLocalEgress("", false) })
+
+	config := &business.APIDatasourceConfig{
+		BaseURL:        stub.URL,
+		CredentialKind: business.APICredentialKindOAuth2,
+		OAuth2: &business.APIOAuth2Config{
+			Grant:    business.OAuth2ClientCredentials,
+			TokenURL: stub.URL + "/token",
+			ClientID: "example-client",
+		},
+	}
+	source, err := svc.AddSource(testCtx, actor, business.AddSourceInput{OrgID: org, Provider: business.DatasourceProviderAPI, CollectionLabel: "Acme", API: config, Credential: clientSecret})
+	require.NoError(t, err)
+
+	// No allowance: the public-address guard refuses the token exchange, and
+	// the stub never sees a request.
+	require.NoError(t, apisource.ConfigureLocalEgress("", true))
+	copy := *source
+	_, err = svc.ResolveDatasourceTokenForTest(testCtx, &copy)
+	require.Error(t, err, "loopback must stay unreachable without an allowance")
+	require.Zero(t, tokenCalls.Load())
+
+	require.NoError(t, apisource.ConfigureLocalEgress(stub.Listener.Addr().String(), true))
+	declaration := connectortest.OperationDeclaration()
+	requestCtx := auth.WithVerifiedDatabaseIdentity(testCtx, actor, org)
+	grantOperationBoundary(t, requestCtx, actor, org, source.BoundaryNodeID)
+	_, err = svc.DeclareSourceOperations(requestCtx, actor, org, source.ID, []operations.Declaration{declaration})
+	require.NoError(t, err)
+	receiptStore, closeStore, err := testStore.NewEffectReceipts()
+	require.NoError(t, err)
+	defer closeStore()
+	require.NoError(t, svc.ConfigureSourceOperationReceipts(receiptStore))
+	_, guard := svc.SourceOperationReceipts()
+	const effect, input = "local-egress", `{"id":"a"}`
+	request := &gen.InvokeSourceOperationRequest{OrgId: org, SourceId: source.ID, Operation: declaration.Name, InputJson: input, EffectId: effect}
+	ctx := business.WithSourceOperationRecheck(requestCtx, func(ctx context.Context) error {
+		_, _, err := svc.SourceOperationAuthority(ctx, actor, org, source.ID, declaration.Name, effect, false)
+		return err
+	})
+	response, err := guard.Handle(ctx, business.SourceOperationMethod, effect, request, func(ctx context.Context) (proto.Message, error) {
+		var response *gen.InvokeSourceOperationResponse
+		_, err := svc.InvokeSourceOperation(ctx, actor, org, source.ID, declaration.Name, []byte(input), func(ctx context.Context, result *business.SourceOperationResult) error {
+			response = &gen.InvokeSourceOperationResponse{OutputJson: string(result.Output), Receipt: &gen.SourceOperationReceipt{EffectId: effect, CommittedAt: result.CommittedAt.UTC().Format(time.RFC3339Nano), Status: "committed", ProviderStatus: uint32(result.ProviderStatus), OutputJson: string(result.Output)}}
+			return svc.RecordSourceOperationReceipt(ctx, response)
+		})
+		return response, err
+	})
+	require.NoError(t, err, "the listed stub must be reachable for both the token exchange and the operation call")
+	require.Equal(t, "{}", response.(*gen.InvokeSourceOperationResponse).OutputJson)
+	require.EqualValues(t, 1, tokenCalls.Load(), "one client_credentials exchange")
+	require.EqualValues(t, 1, apiCalls.Load(), "one operation call, with the exchanged token")
+
+	// Withdrawn, the same source cannot reach the stub again: a fresh exchange
+	// on a second source is refused before it is sent.
+	require.NoError(t, apisource.ConfigureLocalEgress("", true))
+	second, err := svc.AddSource(testCtx, actor, business.AddSourceInput{OrgID: org, Provider: business.DatasourceProviderAPI, CollectionLabel: "Acme", API: config, Credential: clientSecret})
+	require.NoError(t, err)
+	copy = *second
+	_, err = svc.ResolveDatasourceTokenForTest(testCtx, &copy)
+	require.Error(t, err)
+	require.EqualValues(t, 1, tokenCalls.Load(), "a withdrawn allowance must not reach the stub")
+}

module/services/accounts/code/pkg/business/webhook_security.go

@@ -155,8 +155,9 @@ func isAmbiguousNumericHost(host string) bool {
 }
 
 // resolvePublic resolves host and requires every answer to be on the public
-// internet, as auth.IsPublicDestination judges it: the one classifier every
-// tenant-directed egress in this host shares. An address literal is judged as
+// internet, as auth.IsPublicDestination judges it: the classifier the host's
+// guarded egress paths share (auth/public_address.go lists them, and the OIDC
+// clients that have no guard). An address literal is judged as
 // written, zone included, and only unmapped once it has been admitted.
 func (p *WebhookEndpointPolicy) resolvePublic(ctx context.Context, host string) ([]netip.Addr, error) {
 	if addr, err := netip.ParseAddr(host); err == nil {

module/services/accounts/code/pkg/datasource/apisource/client.go

@@ -10,8 +10,10 @@
 // on the initial request and on any redirect, checked against the exact IP
 // being connected to (not the hostname) so DNS rebinding cannot slip past the
 // check. "Not on the public internet" is auth.IsPublicDestination, read from
-// the IANA special-purpose registries: the one classifier every tenant-directed
-// egress in this host answers through.
+// the IANA special-purpose registries: the classifier the host's guarded egress
+// paths share (auth/public_address.go lists them).
+// The one exception is a local runtime's explicit list of loopback addresses
+// (local_egress.go), refused at boot anywhere else.
 package apisource
 
 import (
@@ -93,7 +95,9 @@ func New(cfg Config, credential string) *Client {
 
 // guardedClient builds an HTTP client that refuses connections to non-public
 // addresses, on the initial request and every redirect. Both the fetch client
-// and the OAuth token exchange dial tenant-supplied hosts, so both use it.
+// and the OAuth token exchange dial tenant-supplied hosts, so both use it. A
+// local runtime's explicit allowance (local_egress.go) is consulted first and
+// admits only the exact addresses it lists.
 func guardedClient() *http.Client {
 	transport := &http.Transport{
 		// These clients make a single exchange. An idle connection is neither
@@ -102,7 +106,7 @@ func guardedClient() *http.Client {
 		DisableKeepAlives: true,
 		DialContext: (&net.Dialer{
 			Timeout: 10 * time.Second,
-			Control: dialControl,
+			Control: withLocalEgress(dialControl),
 		}).DialContext,
 	}
 	return &http.Client{