diff --git a/AGENTS.md b/AGENTS.md index 81c8fc8..07801a0 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -84,7 +84,8 @@ repository is the runtime that cost was measured against. to core, the SDK or the host. 4. **Never hardcode what the system resolves.** This package resolves every address, port and secret through the SDK; the single `localhost` literal in - it is the public URL built from the *resolved* port. Both token-exchange URLs + it is the self upstream's listen-address fallback, built from the *resolved* + port and refused by `validate()` in a deployed runtime context. Both token-exchange URLs are derived from their register URLs by swapping the path suffix, which is why an override that drops the documented suffix cannot be paired and is refused at boot rather than guessed at. If you are typing an address, a port diff --git a/README.md b/README.md index 3edd125..942efdd 100644 --- a/README.md +++ b/README.md @@ -48,7 +48,7 @@ the SDK-resolved value is the default. | What | SDK resolution | Env override (default) | |---|---|---| | Own listen port | `codefly.For(ctx).Endpoint("http").NetworkInstance()` — the Codefly-assigned port, not a fixed default | `PORT` | -| Public URL | `http://localhost:` | `PUBLIC_URL` | +| Public URL | none — the manifest is then registered as the root-relative path `/assets/mf-manifest.json` on this backend (see [Where the host reaches the solution](#where-the-host-reaches-the-solution)) | `PUBLIC_URL` | | Gateway URL (auth-gateway `rest`) | resolved by role — the single module owning the `auth-gateway` `rest`/`rest` endpoint, discovered from the injected carriers (or the workspace, run locally) | `GATEWAY_URL` | | Host frontend URL | resolved by role — the single module owning the `frontend` `http`/`http` endpoint, discovered the same way | — (feeds the host register URL) | | Host register URL | `/api/solutions/register` | `HOST_REGISTER_URL` | @@ -59,8 +59,9 @@ the SDK-resolved value is the default. | Internal-auth token | `codefly.For(ctx).WorkspaceSecret("internal-auth", "CODEFLY_INTERNAL_TOKEN")` — the namespaced secret Codefly injects | `CODEFLY_INTERNAL_TOKEN` | | Solution registration secret | `codefly.For(ctx).WorkspaceSecret("solution-registration", "SECRET")` — see [Self-registration](#self-registration) | `CODEFLY__SOLUTION_REGISTRATION_SECRET` | | Registration beat interval | `15s` | `CODEFLY__SOLUTION_REGISTRATION_INTERVAL` | -| Self upstream | `` | `SELF_UPSTREAM` | -| MF assets dir | `../fe-remote/dist` | `ASSETS_DIR` | +| Self upstream | the reachable self endpoint Codefly injects as `CODEFLY__SELF_ENDPOINT______HTTP__HTTP` (core ≥ v0.5.6); without it, the listen address `http://localhost:` | `SELF_UPSTREAM` | +| Deployed or local | `CODEFLY__RUNTIME_CONTEXT`, injected by Codefly: `native`/`nix`/`container`/`free` (or unset) is a local run, anything else (a GitOps render's `kubernetes`, core ≥ v0.5.6) a deployment | — | +| MF assets | `Manifest.Assets` when set (see below), else the `../fe-remote/dist` directory | `ASSETS_DIR` (directory only) | The host it plugs into is named by Codefly-convention **service roles**, not by its workspace module name: the runtime discovers the single module that owns each @@ -75,6 +76,49 @@ exposes the same role — otherwise an ambiguous match resolves to nothing and t runtime fails loud at boot rather than picking a host arbitrarily. Concrete addresses always come from the SDK. +### Where the host reaches the solution + +The two registrations carry two different addresses, for two different +callers, and neither is this process's listen address: + +- **`manifestUrl`** (host registration) is loaded by the **viewer's browser**. + With no `PUBLIC_URL` it is the root-relative path `/assets/mf-manifest.json` + on this backend, and the host resolves it against the route by which it + reaches the solution — the runtime does not know, and does not encode, the + host's route layout. `PUBLIC_URL` makes it absolute on that origin instead, + for an operator who exposes the solution's assets directly. +- **`upstream`** (gateway registration) is dialled by the **gateway**. It is + the address Codefly injects for reaching this service + (`CODEFLY__SELF_ENDPOINT__…`, beside the `CODEFLY__ENDPOINT__…` carrier that + stays the listen address), and `SELF_UPSTREAM` overrides it. `PUBLIC_URL` no + longer feeds it: the browser's origin is not the gateway's route. + +In a **deployed** runtime context (`CODEFLY__RUNTIME_CONTEXT` outside the local +kinds above) `validate()` refuses to boot when either address is loopback +(`localhost`, `*.localhost`, `127.0.0.0/8`, `::1`, the unspecified address): a +deployed solution that registered its listen address booted, looked healthy, and +was proxied by the gateway to the gateway itself, while the product reported it +as failed to load. The environment's *name* is never consulted — an environment +called `local-dogfood` may well be deployed. + +Until core v0.5.6 is released, the self endpoint is read from that carrier by +name in one helper (`selfEndpoint`), and a cell rendered by an older core has +neither the carrier nor the runtime-context signal: it falls back to the listen +address and is not refused. Both switch to the core/SDK accessors when released. + +### Serving the frontend + +`/assets/` serves the built Module Federation remote, with CORS, from +`Manifest.Assets` when the solution sets it — typically an `embed.FS` narrowed +with `fs.Sub` to the build's output directory — and otherwise from the +`ASSETS_DIR` directory. A solution serving from `Manifest.Assets` reads nothing +from disk, so it runs with a read-only root filesystem. Either way a file whose +name carries a content hash (`123.3f9a1c2b.js`) is served +`public, max-age=31536000, immutable`, and everything else — `mf-manifest.json` +and the remote entry above all, whose names never change — `no-cache`, as is +any response that is not the file (a `404` for a hashed name must not be cached +under the name the next build will serve). + ### Self-registration On boot the runtime self-registers on a 15s heartbeat with **both** the host @@ -121,6 +165,13 @@ registration: it proves no publisher, and a downgrade path would let anything able to answer `404` at that URL turn the publisher-bound credential back into it. +Only a `401`/`403` from the exchange is reported as a credential fault (no +`SOLUTION_REGISTRATION_SECRETS` entry for this id, or a secret that does not +match its digest) — those are the statuses accounts answers after judging the +secret. A `5xx` is reported as `host unavailable (status N), retrying` with the +gateway's own error body, and any other status as a failure with its body; none +of them names the provisioning, and the heartbeat keeps retrying all of them. + A refusal that carries reasons (the host's `409 incompatible_runtime`, say) is logged with them, again whenever the reasons change and not only when the status does — up to a handful of distinct reasons per status, after which the log says diff --git a/deployed_registration_test.go b/deployed_registration_test.go new file mode 100644 index 0000000..8dcf95c --- /dev/null +++ b/deployed_registration_test.go @@ -0,0 +1,307 @@ +package solution + +import ( + "context" + "io" + "net" + "net/http" + "net/http/httptest" + "os" + "path/filepath" + "strconv" + "strings" + "testing" + "testing/fstest" + "time" +) + +// clearSelfEnvironment unsets every carrier these tests key on, so a variable +// exported in the shell running the suite cannot decide an assertion. +func clearSelfEnvironment(t *testing.T) { + t.Helper() + for _, key := range []string{"SELF_UPSTREAM", "PUBLIC_URL", "CODEFLY__RUNTIME_CONTEXT", + "CODEFLY__MODULE", "CODEFLY__SERVICE", "CODEFLY__ENVIRONMENT", + "CODEFLY__SELF_ENDPOINT__LASTLOGIN_GO__BACKEND__HTTP__HTTP"} { + t.Setenv(key, "") + } +} + +// TestSelfUpstreamIsTheReachableSelfEndpoint proves the upstream registered +// with the gateway is the address Codefly injects for reaching this service, +// not its own listen address. A deployed solution registered +// "http://localhost:8080" — its listen address, via the public URL default — +// and the gateway, dialling its own localhost, proxied the solution to itself. +func TestSelfUpstreamIsTheReachableSelfEndpoint(t *testing.T) { + const reachable = "http://backend.solutions-lastlogin.svc.cluster.local:8080" + for _, tc := range []struct { + name string + carrier string + override string + want string + }{ + {name: "the injected self endpoint", carrier: reachable, want: reachable}, + {name: "the listen address when none is injected", want: "http://localhost:8080"}, + {name: "an explicit override over the carrier", carrier: reachable, override: "http://upstream.example.com:9000/", want: "http://upstream.example.com:9000"}, + } { + t.Run(tc.name, func(t *testing.T) { + clearSelfEnvironment(t) + t.Setenv("PORT", "8080") + t.Setenv("CODEFLY__MODULE", "lastlogin-go") + t.Setenv("CODEFLY__SERVICE", "backend") + // The key is normalised as CODEFLY__ENDPOINT__ keys are: upper + // case, dashes to underscores. + t.Setenv("CODEFLY__SELF_ENDPOINT__LASTLOGIN_GO__BACKEND__HTTP__HTTP", tc.carrier) + t.Setenv("SELF_UPSTREAM", tc.override) + cfg := loadConfig(context.Background(), "lastlogin-go") + if cfg.selfUpstream != tc.want { + t.Fatalf("selfUpstream = %q, want %q", cfg.selfUpstream, tc.want) + } + // PUBLIC_URL is the browser's origin, not the gateway's upstream: + // it no longer feeds the registered upstream at all. + t.Setenv("PUBLIC_URL", "https://solutions.example.com") + if got := loadConfig(context.Background(), "lastlogin-go").selfUpstream; got != tc.want { + t.Fatalf("with PUBLIC_URL set, selfUpstream = %q, want %q", got, tc.want) + } + }) + } +} + +// TestGatewayRegistrationCarriesTheReachableUpstream proves the carrier reaches +// the wire: the body the gateway is sent names the injected self endpoint. +func TestGatewayRegistrationCarriesTheReachableUpstream(t *testing.T) { + const reachable = "http://backend.solutions-lastlogin.svc.cluster.local:8080" + bodies := make(chan string, 16) + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + if r.URL.Path == solutionRegisterPath { + body, _ := io.ReadAll(r.Body) + send(bodies, string(body)) + } + answeringTheExchange(w, r) + })) + defer srv.Close() + ln, err := net.Listen("tcp", "127.0.0.1:0") + if err != nil { + t.Fatal(err) + } + s := New(Manifest{ID: "lastlogin-go", Title: "Last Login"}) + s.cfg = config{ + port: strconv.Itoa(ln.Addr().(*net.TCPAddr).Port), + hostRegisterURL: srv.URL + "/api/solutions/register", + gatewayRegisterURL: srv.URL + solutionRegisterPath, + moduleRegisterURL: srv.URL + moduleRegisterPath, + moduleTokenURL: srv.URL + moduleRegistrationTokenPath, + solutionTokenURL: srv.URL + solutionRegistrationTokenPath, + internalToken: internalTokenTest, + solutionSecret: "s3cret", + selfUpstream: reachable, + } + ctx, cancel := context.WithCancel(context.Background()) + done := make(chan struct{}) + go func() { _ = s.serve(ctx, ln); close(done) }() + defer func() { cancel(); <-done }() + + select { + case body := <-bodies: + if !strings.Contains(body, `"upstream":"`+reachable+`"`) { + t.Fatalf("gateway registration body = %s, want upstream %q", body, reachable) + } + case <-time.After(5 * time.Second): + t.Fatal("no gateway registration within timeout") + } +} + +// TestValidateRefusesLoopbackInADeployedRuntime proves a deployed process +// cannot boot registering an address only it can reach — and that the signal +// is the explicit runtime context, never the environment's name. +func TestValidateRefusesLoopbackInADeployedRuntime(t *testing.T) { + base := config{ + port: "8080", + gatewayURL: "http://gateway:42152", + hostRegisterURL: "http://frontend:21931/api/solutions/register", + gatewayRegisterURL: "http://gateway:42152/solutions/_register", + moduleRegisterURL: "http://gateway:42152/modules/_register", + moduleTokenURL: "http://gateway:42152/modules/_registration-token", + solutionTokenURL: "http://gateway:42152/solutions/_registration-token", + solutionSecret: "s3cret", + selfUpstream: "http://backend.solutions-lastlogin.svc.cluster.local:8080", + } + loopbacks := []string{"http://localhost:8080", "http://127.0.0.1:8080", "http://[::1]:8080", + "http://0.0.0.0:8080", "http://app.localhost:8080", "http://LOCALHOST.:8080"} + + for _, upstream := range loopbacks { + c := base + c.runtimeContext = "kubernetes" + c.selfUpstream = upstream + err := c.validate() + if err == nil || !strings.Contains(err.Error(), "self upstream") || !strings.Contains(err.Error(), selfEndpointCarrierPrefix) { + t.Errorf("deployed with upstream %q: validate() = %v, want a refusal naming the self upstream and its carrier", upstream, err) + } + // Every runtime context `codefly run` uses locally accepts it: there + // the gateway shares the machine. + for _, local := range []string{"", "native", "nix", "container", "free"} { + c.runtimeContext = local + if err := c.validate(); err != nil { + t.Errorf("runtime context %q with upstream %q: validate() = %v, want nil", local, upstream, err) + } + } + } + + c := base + c.runtimeContext = "kubernetes" + c.publicURL = "http://localhost:8080" + if err := c.validate(); err == nil || !strings.Contains(err.Error(), "PUBLIC_URL") { + t.Errorf("deployed with a loopback PUBLIC_URL: validate() = %v, want a refusal naming PUBLIC_URL", err) + } + c.publicURL = "https://solutions.example.com" + if err := c.validate(); err != nil { + t.Errorf("deployed with a reachable upstream and public URL: validate() = %v, want nil", err) + } + c.selfUpstream = "/relative" + if err := c.validate(); err == nil || !strings.Contains(err.Error(), "SELF_UPSTREAM") { + t.Errorf("relative upstream: validate() = %v, want a refusal naming SELF_UPSTREAM", err) + } +} + +// TestLoopbackRefusalIgnoresTheEnvironmentName proves the environment's name +// decides nothing: an environment named like a local one still refuses a +// loopback upstream when the runtime context says it is deployed, and one +// named like a deployment accepts it when nothing says so. +func TestLoopbackRefusalIgnoresTheEnvironmentName(t *testing.T) { + for _, tc := range []struct { + environment, runtimeContext string + refused bool + }{ + {environment: "local-dogfood", runtimeContext: "kubernetes", refused: true}, + {environment: "local", runtimeContext: "kubernetes", refused: true}, + {environment: "staging", runtimeContext: "native", refused: false}, + {environment: "local-dogfood", runtimeContext: "container", refused: false}, + } { + clearSelfEnvironment(t) + t.Setenv("PORT", "8080") + t.Setenv("GATEWAY_URL", "http://gateway:42152") + t.Setenv("HOST_REGISTER_URL", "http://frontend:21931/api/solutions/register") + t.Setenv(SolutionRegistrationSecretEnvironmentVariable, "s3cret") + t.Setenv("CODEFLY__ENVIRONMENT", tc.environment) + t.Setenv("CODEFLY__RUNTIME_CONTEXT", tc.runtimeContext) + err := loadConfig(context.Background(), "lastlogin-go").validate() + if refused := err != nil && strings.Contains(err.Error(), "loopback"); refused != tc.refused || (err != nil && !refused) { + t.Errorf("environment %q, runtime context %q: validate() = %v, want refused=%v", + tc.environment, tc.runtimeContext, err, tc.refused) + } + } +} + +// TestAssetsServeFromTheManifestFS proves a solution that ships its frontend +// in Manifest.Assets serves it with nothing on disk — ASSETS_DIR points +// nowhere, as on a pod with a read-only root filesystem — and that the +// root-relative manifestUrl it registers resolves to a file that FS holds. +func TestAssetsServeFromTheManifestFS(t *testing.T) { + assets := fstest.MapFS{ + "mf-manifest.json": {Data: []byte(`{"name":"lastlogin"}`)}, + "static/js/async/123.3f9a1c2b.js": {Data: []byte(`export default 1`)}, + } + base := bootWithAssets(t, Manifest{ID: "lastlogin-go", Assets: assets}, filepath.Join(t.TempDir(), "absent")) + + manifestURL := frontendManifestURL(t, base) + if manifestURL != federationManifestPath { + t.Fatalf("manifestUrl = %q, want %q", manifestURL, federationManifestPath) + } + resp := get(t, base+manifestURL) + if resp.status != http.StatusOK || resp.body != `{"name":"lastlogin"}` { + t.Fatalf("GET %s = %d %q, want the manifest from Manifest.Assets", manifestURL, resp.status, resp.body) + } + if !strings.HasPrefix(resp.header.Get("content-type"), "application/json") { + t.Errorf("manifest content-type = %q, want application/json", resp.header.Get("content-type")) + } + if resp.header.Get("access-control-allow-origin") != "*" { + t.Errorf("manifest served without CORS") + } + if got := resp.header.Get("cache-control"); got != "no-cache" { + t.Errorf("manifest cache-control = %q, want no-cache: its name is fixed, so a cached copy outlives a redeploy", got) + } + + chunk := get(t, base+"/assets/static/js/async/123.3f9a1c2b.js") + if chunk.status != http.StatusOK || !strings.Contains(chunk.header.Get("content-type"), "javascript") { + t.Fatalf("hashed chunk = %d %q", chunk.status, chunk.header.Get("content-type")) + } + if got := chunk.header.Get("cache-control"); got != "public, max-age=31536000, immutable" { + t.Errorf("hashed chunk cache-control = %q, want immutable", got) + } + + missing := get(t, base+"/assets/static/js/async/456.0badc0de.js") + if missing.status != http.StatusNotFound { + t.Fatalf("missing chunk = %d, want 404", missing.status) + } + if got := missing.header.Get("cache-control"); got != "no-cache" { + t.Errorf("missing hashed chunk cache-control = %q, want no-cache: a 404 must not be cached for a year under the name the next build serves", got) + } +} + +// TestAssetsServeFromAssetsDirWithoutAnFS proves the directory path is kept +// for a solution that sets no Manifest.Assets, with the same cache policy. +func TestAssetsServeFromAssetsDirWithoutAnFS(t *testing.T) { + dir := t.TempDir() + if err := os.WriteFile(filepath.Join(dir, "mf-manifest.json"), []byte(`{"name":"disk"}`), 0o644); err != nil { + t.Fatal(err) + } + base := bootWithAssets(t, Manifest{ID: "lastlogin-go"}, dir) + resp := get(t, base+federationManifestPath) + if resp.status != http.StatusOK || resp.body != `{"name":"disk"}` { + t.Fatalf("GET manifest = %d %q, want the file from ASSETS_DIR", resp.status, resp.body) + } + if got := resp.header.Get("cache-control"); got != "no-cache" { + t.Errorf("manifest cache-control = %q, want no-cache", got) + } +} + +func TestAssetCacheControl(t *testing.T) { + for name, want := range map[string]string{ + "/mf-manifest.json": "no-cache", + "/remoteEntry.js": "no-cache", + "/static/js/123.3f9a1c2b.js": "public, max-age=31536000, immutable", + "/static/css/index-3f9a1c2b0d.css": "public, max-age=31536000, immutable", + "/static/js/lib-react.js": "no-cache", + "/static/js/deadbeef.js": "no-cache", + } { + if got := assetCacheControl(name); got != want { + t.Errorf("assetCacheControl(%q) = %q, want %q", name, got, want) + } + } +} + +type response struct { + status int + header http.Header + body string +} + +func get(t *testing.T, target string) response { + t.Helper() + resp, err := http.Get(target) + if err != nil { + t.Fatalf("GET %s: %v", target, err) + } + defer resp.Body.Close() + body, _ := io.ReadAll(resp.Body) + return response{status: resp.StatusCode, header: resp.Header, body: string(body)} +} + +// bootWithAssets serves m on an ephemeral port with assetsDir as ASSETS_DIR +// and returns its base URL. Registration targets a closed port: these tests +// are about what the solution serves, not whom it registers with. +func bootWithAssets(t *testing.T, m Manifest, assetsDir string) string { + t.Helper() + ln, err := net.Listen("tcp", "127.0.0.1:0") + if err != nil { + t.Fatal(err) + } + s := New(m) + s.cfg = config{port: strconv.Itoa(ln.Addr().(*net.TCPAddr).Port), assetsDir: assetsDir} + ctx, cancel := context.WithCancel(context.Background()) + done := make(chan struct{}) + go func() { _ = s.serve(ctx, ln); close(done) }() + t.Cleanup(func() { cancel(); <-done }) + base := "http://" + ln.Addr().String() + waitFor(t, "health", func() bool { return getStatus(t, base+"/health") == http.StatusOK }) + return base +} diff --git a/solution.go b/solution.go index 7b9e808..d74d1b5 100644 --- a/solution.go +++ b/solution.go @@ -17,12 +17,15 @@ import ( "errors" "fmt" "io" + "io/fs" "log" "math/rand/v2" "net" "net/http" "net/url" "os" + "path" + "regexp" "strconv" "strings" "sync" @@ -55,6 +58,15 @@ type Manifest struct { // host's registry projects them so a client can discover, without a table // of its own, which solutions offer it something. Surfaces []Surface + // Assets, when set, is what /assets/ serves: the built Module Federation + // remote (mf-manifest.json at its root, and every chunk it names), usually + // embedded in the binary with go:embed and narrowed with fs.Sub to the + // build's output directory. A solution that ships its frontend this way + // reads nothing from disk to serve it, so it runs with a read-only root + // filesystem and cannot drift from the binary it was built with. + // + // When nil, /assets/ serves the ASSETS_DIR directory, as before. + Assets fs.FS } // Surface is one offering a solution makes inside a client application. @@ -238,6 +250,10 @@ type config struct { // one was not usable. registrationInterval time.Duration registrationIntervalErr error + // runtimeContext is the kind of runtime Codefly says this process runs + // under (CODEFLY__RUNTIME_CONTEXT), the explicit signal validate() uses to + // tell a deployed process from a local one. Empty when nothing injected it. + runtimeContext string // environmentLoadErr is the failure, if any, of loading Codefly's injected // carriers. Every SDK-resolved value below is empty when that load failed, // so validate() must say so rather than report each empty value as @@ -407,10 +423,10 @@ func loadConfig(ctx context.Context, id string) config { port = listenPort(self) } } - public := env("PUBLIC_URL", "") - if public == "" { - public = "http://localhost:" + port - } + // Empty unless an operator set one: the manifest is then registered + // root-relative (see frontendManifestURL), never on this process's own + // loopback listen address. + public := strings.TrimRight(env("PUBLIC_URL", ""), "/") // Host endpoints, resolved via the SDK (no localhost:port literals). gatewayURL := strings.TrimRight(env("GATEWAY_URL", resolveGateway(ctx, hostModule, hostGateway)), "/") @@ -443,16 +459,100 @@ func loadConfig(ctx context.Context, id string) config { moduleRegisterURL: moduleRegisterURL, moduleTokenURL: env("GATEWAY_MODULE_REGISTRATION_TOKEN_URL", siblingURL(moduleRegisterURL, moduleRegisterPath, moduleRegistrationTokenPath)), solutionTokenURL: env("GATEWAY_SOLUTION_REGISTRATION_TOKEN_URL", siblingURL(gatewayRegisterURL, solutionRegisterPath, solutionRegistrationTokenPath)), - selfUpstream: env("SELF_UPSTREAM", public), + selfUpstream: strings.TrimRight(env("SELF_UPSTREAM", selfUpstream(port)), "/"), assetsDir: env("ASSETS_DIR", "../fe-remote/dist"), internalToken: token, solutionSecret: solutionRegistrationSecret(ctx), moduleSecrets: parseModuleRegistrationSecrets(env(ModuleRegistrationSecretsEnvironmentVariable, "")), + runtimeContext: strings.TrimSpace(env(resources.RuntimeContextPrefix, "")), } cfg.registrationInterval, cfg.registrationIntervalErr = registrationIntervalFromEnv() return cfg } +// selfEndpointCarrierPrefix names the carrier Codefly injects with this +// service's own endpoint as the rest of the deployment reaches it — its +// in-cluster address, e.g. "http://backend..svc.cluster.local:8080". +// It sits beside the CODEFLY__ENDPOINT__ carrier for the same endpoint, which +// stays what it is: the address this process listens on ("localhost:8080" in a +// rendered cell), right for binding and wrong for anyone else to dial. +// +// PENDING codefly-dev/core v0.5.6, which adds this carrier. It is read here by +// name, in this one helper, only until core/sdk-go release an accessor for it; +// switch selfEndpoint to that accessor when they do. +const selfEndpointCarrierPrefix = "CODEFLY__SELF_ENDPOINT" + +// selfEndpoint is this service's reachable http endpoint from the carrier +// above, or "" when none was injected (a local run, or a render from before +// core v0.5.6). The key is normalised exactly as the CODEFLY__ENDPOINT__ +// carriers are. +func selfEndpoint() string { + module, service := os.Getenv(resources.ModulePrefix), os.Getenv(resources.ServicePrefix) + if module == "" || service == "" { + return "" + } + key := selfEndpointCarrierPrefix + "__" + resources.EndpointAsEnvironmentVariableKeyBase(&resources.EndpointInformation{ + Module: module, Service: service, Name: "http", API: "http", + }) + return strings.TrimSpace(os.Getenv(key)) +} + +// selfUpstream is the upstream this solution registers with the gateway: the +// address the gateway dials to proxy to it. It is the reachable self endpoint +// when Codefly injected one, and otherwise this process's own listen address — +// correct only when the gateway runs on the same machine, which is why +// validate() refuses a loopback upstream in a deployed runtime context rather +// than letting the gateway proxy the solution to itself. +// +// It used to be the public URL, which defaulted to that same loopback listen +// address, so every deployed solution registered "http://localhost:8080" and +// the gateway, dialling its own localhost, proxied the solution to itself. +func selfUpstream(port string) string { + if self := selfEndpoint(); self != "" { + return self + } + if port == "" { + return "" + } + return "http://localhost:" + port +} + +// deployedRuntimeContext reports whether Codefly says this process runs in a +// deployment rather than on a developer machine. The signal is explicit — +// CODEFLY__RUNTIME_CONTEXT, which core's GitOps render injects (e.g. +// "kubernetes") — and never the environment name: an environment called +// "local-dogfood" or "staging" says nothing about where the process runs. +// Every runtime context `codefly run` uses on a developer machine is local; +// any other declared context is a deployment, so a new deployed kind is +// covered without a change here. Nothing declared means not deployed. +// +// PENDING codefly-dev/core v0.5.6, which injects the signal into renders: +// until a cell is rendered with it, this reports false there too. +func deployedRuntimeContext(kind string) bool { + switch strings.ToLower(kind) { + case "", resources.RuntimeContextNative, resources.RuntimeContextNix, + resources.RuntimeContextContainer, resources.RuntimeContextFree: + return false + } + return true +} + +// loopbackURL reports whether raw names this machine: localhost (or a name +// under .localhost), a loopback IP, or the unspecified address. Such a URL is +// reachable only from the process's own host. +func loopbackURL(raw string) bool { + u, err := url.Parse(raw) + if err != nil { + return false + } + host := strings.ToLower(strings.TrimSuffix(u.Hostname(), ".")) + if host == "localhost" || strings.HasSuffix(host, ".localhost") { + return true + } + ip := net.ParseIP(host) + return ip != nil && (ip.IsLoopback() || ip.IsUnspecified()) +} + // RegistrationIntervalEnvironmentVariable sets how long a registration // heartbeat waits between beats, as a Go duration ("30s", "2m"). // @@ -543,6 +643,7 @@ func (c config) validate() error { "module register URL": c.moduleRegisterURL, "module token URL": c.moduleTokenURL, "solution token URL": c.solutionTokenURL, + "self upstream": c.selfUpstream, } // A token URL is not resolved from the SDK at all: it is derived from its // register URL by siblingURL, which yields "" for a base it cannot pair. @@ -564,9 +665,28 @@ func (c config) validate() error { if hint, ok := unpairable[name]; ok && raw == "" { return fmt.Errorf("unresolved %s: %s", name, hint) } + if name == "self upstream" { + // Not a host endpoint: it is this solution's own address, from + // its own endpoint carriers or an explicit SELF_UPSTREAM. + return fmt.Errorf("unresolved self upstream %q: SELF_UPSTREAM must be an absolute URL the gateway can dial, or unset so the address Codefly injects for this service is used", raw) + } return fmt.Errorf("unresolved %s %q: the SDK could not resolve the host endpoint and no explicit override was set", name, raw) } } + if deployedRuntimeContext(c.runtimeContext) { + // A loopback address registered from a deployment is reachable by no + // one: the gateway dialling "localhost" proxies the solution to + // itself, and a browser loading a manifest from "localhost" loads it + // from the viewer's own machine. Both used to boot and look healthy. + if loopbackURL(c.selfUpstream) { + return fmt.Errorf("self upstream %q is a loopback address in the deployed runtime context %q: the gateway would proxy this solution to itself. Codefly injects the reachable address as %s______HTTP__HTTP (core >= v0.5.6); set SELF_UPSTREAM only for a deployment the resolver cannot see", + c.selfUpstream, c.runtimeContext, selfEndpointCarrierPrefix) + } + if c.publicURL != "" && loopbackURL(c.publicURL) { + return fmt.Errorf("PUBLIC_URL %q is a loopback address in the deployed runtime context %q: no browser but this machine's could load the manifest from it. Unset it to register the manifest relative to this backend, or set the origin browsers actually reach", + c.publicURL, c.runtimeContext) + } + } if c.registrationIntervalErr != nil { return c.registrationIntervalErr } @@ -657,8 +777,7 @@ func (s *Server) serve(ctx context.Context, ln net.Listener) error { for path, handler := range s.handlers { mux.HandleFunc(path, withCORS(s.wrapRequest(handler))) } - mux.Handle("/assets/", http.StripPrefix("/assets/", - withCORSHandler(http.FileServer(http.Dir(s.cfg.assetsDir))))) + mux.Handle("/assets/", http.StripPrefix("/assets/", withCORSHandler(s.assetsHandler()))) manifestBody, _ := json.Marshal(s.manifestMap()) upstreamBody, _ := json.Marshal(map[string]string{"id": s.manifest.ID, "upstream": s.cfg.selfUpstream}) @@ -781,7 +900,7 @@ func (s *Server) manifestMap() map[string]any { "nav": map[string]any{"title": s.manifest.Title, "path": "/s/" + s.manifest.ID, "order": s.manifest.Order}, "frontend": map[string]any{ "type": "module-federation", - "manifestUrl": s.cfg.publicURL + "/assets/mf-manifest.json", + "manifestUrl": s.frontendManifestURL(), "exposedModule": s.manifest.ExposedModule, "hostContract": solutionHostContractMajor, "reactRange": "^19", @@ -824,6 +943,92 @@ func (s *Server) manifestMap() map[string]any { return m } +// assetsHandler serves the frontend build under /assets/ (the prefix already +// stripped): from Manifest.Assets when the solution ships one, else from the +// ASSETS_DIR directory. Both go through the standard file server, so content +// types and range/conditional handling are the same whichever source serves. +func (s *Server) assetsHandler() http.Handler { + var files http.Handler + if s.manifest.Assets != nil { + files = http.FileServerFS(s.manifest.Assets) + } else { + files = http.FileServer(http.Dir(s.cfg.assetsDir)) + } + return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + files.ServeHTTP(&assetCacheWriter{ResponseWriter: w, cacheControl: assetCacheControl(r.URL.Path)}, r) + }) +} + +// assetCacheWriter stamps an asset's cache policy on the response, but only on +// one that delivers or confirms the asset. A 404 for a hashed chunk — asked of +// an old replica mid-rollout — must not be cached for a year under the name +// the new build will serve it at. +type assetCacheWriter struct { + http.ResponseWriter + cacheControl string + wrote bool +} + +func (w *assetCacheWriter) WriteHeader(status int) { + if !w.wrote { + w.wrote = true + policy := w.cacheControl + if status >= 300 && status != http.StatusNotModified { + policy = "no-cache" + } + w.Header().Set("cache-control", policy) + } + w.ResponseWriter.WriteHeader(status) +} + +func (w *assetCacheWriter) Write(b []byte) (int, error) { + if !w.wrote { + w.WriteHeader(http.StatusOK) + } + return w.ResponseWriter.Write(b) +} + +// contentHashedAsset matches a file name carrying a build content hash — a +// run of at least eight hex digits between separators, as the bundlers emit +// ("123.3f9a1c2b.js", "index-3f9a1c2b0d.css"). +var contentHashedAsset = regexp.MustCompile(`[.-][0-9a-f]{8,}\.[A-Za-z0-9]+$`) + +// assetCacheControl is the caching a served asset may have. A content-hashed +// file never changes under its name, so it is cached for good. Everything +// else — above all mf-manifest.json and the remote entry, whose names are +// fixed so the host can find them — must be revalidated on every load, or a +// browser keeps a manifest naming chunks the redeployed solution no longer +// serves. +func assetCacheControl(name string) string { + if contentHashedAsset.MatchString(path.Base(name)) { + return "public, max-age=31536000, immutable" + } + return "no-cache" +} + +// federationManifestPath is where this runtime serves its Module Federation +// manifest, relative to its own backend: the /assets/ file server in serve. +const federationManifestPath = "/assets/mf-manifest.json" + +// frontendManifestURL is the manifestUrl this solution registers. With an +// explicit PUBLIC_URL it is absolute on that origin — an operator who exposes +// the solution's assets directly said where. Without one it is the path on the +// solution's own backend, root-relative, and the host resolves it against the +// route by which it reaches this solution. +// +// It used to be absolute on "http://localhost:", which is this process's +// own listen address: true in a browser on the developer's machine and in no +// other browser anywhere, so every deployed solution registered a manifest the +// product could not load. The runtime cannot build a better absolute URL, +// because the origin a browser reaches this solution through is the host's — +// and naming the host's route layout here would couple every solution to it. +func (s *Server) frontendManifestURL() string { + if s.cfg.publicURL != "" { + return strings.TrimRight(s.cfg.publicURL, "/") + federationManifestPath + } + return federationManifestPath +} + func (s *Server) handleManifest(w http.ResponseWriter, _ *http.Request) { writeJSON(w, http.StatusOK, s.manifestMap()) } @@ -1139,6 +1344,17 @@ func oneLine(s string) string { }, s) } +// exchangeFailureBody is what the gateway said alongside a failed exchange, +// bounded and flattened to one log line: the host's own words are the only +// clue to which of its dependencies is down. "no body" when it said nothing. +func exchangeFailureBody(resp *http.Response) string { + raw, _ := io.ReadAll(io.LimitReader(resp.Body, refusalDetailMaxBytes)) + if body := strings.TrimSpace(oneLine(string(raw))); body != "" { + return body + } + return "no body" +} + // drainAndClose consumes what is left of a response body before closing it, so // the connection returns to the pool instead of being dropped and re-dialled on // the next beat. Bounded: a body larger than this is not worth reading to keep @@ -1614,11 +1830,26 @@ func (c *solutionCredential) exchange(ctx context.Context) (string, time.Time, e // token proves no publisher — so the beat fails and names the route. return "", time.Time{}, fmt.Errorf("the solution registration exchange at %s answered 404: this host does not serve publisher-bound registration (module-saas-starter < v0.0.61 is not supported) or the URL is wrong (%s, or the %s it is derived from)", c.tokenURL, "GATEWAY_SOLUTION_REGISTRATION_TOKEN_URL", "GATEWAY_REGISTER_URL") - default: + case http.StatusUnauthorized, http.StatusForbidden: // accounts answers an undeclared id and a wrong secret identically, and // the gateway relays that; naming both here is what the reader needs. + // Only these two statuses are a verdict on the credential: they are the + // ones accounts answers after reading the secret. return "", time.Time{}, fmt.Errorf("registration token exchange for solution %q rejected (status %d): the host declares no %s entry for this id, or the provisioned secret does not match its digest", c.id, resp.StatusCode, "SOLUTION_REGISTRATION_SECRETS") + default: + // Anything else was never a judgement of the secret. Blaming the + // provisioning for it sent an operator to re-provision a secret that + // was correct while the gateway could not reach accounts (a 502 relayed + // as "the provisioned secret does not match its digest"). A 5xx is the + // host being unavailable; the heartbeat retries it on its short cap, so + // say that, with what the gateway said. + if resp.StatusCode >= 500 { + return "", time.Time{}, fmt.Errorf("registration token exchange for solution %q: host unavailable (status %d), retrying: %s", + c.id, resp.StatusCode, exchangeFailureBody(resp)) + } + return "", time.Time{}, fmt.Errorf("registration token exchange for solution %q failed (status %d), retrying: %s", + c.id, resp.StatusCode, exchangeFailureBody(resp)) } var issued struct { Token string `json:"token"` diff --git a/solution_registration_test.go b/solution_registration_test.go index 7133b77..87bfa2a 100644 --- a/solution_registration_test.go +++ b/solution_registration_test.go @@ -524,6 +524,7 @@ func TestValidateRequiresASolutionSecret(t *testing.T) { moduleRegisterURL: "http://gateway:42152/modules/_register", moduleTokenURL: "http://gateway:42152/modules/_registration-token", solutionTokenURL: "http://gateway:42152/solutions/_registration-token", + selfUpstream: "http://backend:8080", } err := cfg.validate() if err == nil || !strings.Contains(err.Error(), SolutionRegistrationSecretEnvironmentVariable) || @@ -896,3 +897,103 @@ func TestRegistrationIntervalIsConfigurable(t *testing.T) { }) } } + +// TestSolutionExchangeBlamesTheCredentialOnlyWhenItWasJudged proves only a +// 401/403 from the exchange names the provisioning. A 5xx is the host being +// unavailable — reported with the gateway's own words — and before this every +// status fell through to "no SOLUTION_REGISTRATION_SECRETS entry / digest +// mismatch", sending an operator to re-provision a secret that was correct +// while the gateway could not reach accounts. +func TestSolutionExchangeBlamesTheCredentialOnlyWhenItWasJudged(t *testing.T) { + for _, tc := range []struct { + status int + body string + want []string + credential bool + }{ + {status: http.StatusUnauthorized, credential: true}, + {status: http.StatusForbidden, credential: true}, + {status: http.StatusInternalServerError, body: `{"error":"accounts unavailable"}`, + want: []string{"host unavailable (status 500), retrying", `{"error":"accounts unavailable"}`}}, + {status: http.StatusBadGateway, body: "upstream connect error\nor disconnect", + want: []string{"host unavailable (status 502), retrying", "upstream connect error or disconnect"}}, + {status: http.StatusServiceUnavailable, + want: []string{"host unavailable (status 503), retrying: no body"}}, + {status: http.StatusTooManyRequests, body: "slow down", + want: []string{"failed (status 429), retrying: slow down"}}, + } { + t.Run(strconv.Itoa(tc.status), func(t *testing.T) { + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) { + w.WriteHeader(tc.status) + _, _ = io.WriteString(w, tc.body) + })) + defer srv.Close() + credential := &solutionCredential{tokenURL: srv.URL, id: "lastlogin-go", secret: "s3cret"} + _, _, err := credential.exchange(context.Background()) + if err == nil { + t.Fatalf("status %d: exchange succeeded", tc.status) + } + blames := strings.Contains(err.Error(), "SOLUTION_REGISTRATION_SECRETS") + if blames != tc.credential { + t.Fatalf("status %d: blames the credential = %v, want %v: %v", tc.status, blames, tc.credential, err) + } + for _, want := range tc.want { + if !strings.Contains(err.Error(), want) { + t.Errorf("status %d: error %q does not contain %q", tc.status, err, want) + } + } + }) + } +} + +// TestSolutionRegistrationKeepsRetryingAnUnavailableHost proves a host whose +// exchange answers 5xx is retried, not given up on, and that the log says the +// host is unavailable rather than that the secret is wrong. +func TestSolutionRegistrationKeepsRetryingAnUnavailableHost(t *testing.T) { + buf := &syncBuffer{} + log.SetOutput(buf) + defer log.SetOutput(os.Stderr) + + var exchanges atomic.Int32 + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + if r.URL.Path == solutionRegistrationTokenPath { + exchanges.Add(1) + w.WriteHeader(http.StatusBadGateway) + _, _ = io.WriteString(w, "accounts unreachable") + return + } + w.WriteHeader(http.StatusOK) + })) + defer srv.Close() + + ln, err := net.Listen("tcp", "127.0.0.1:0") + if err != nil { + t.Fatal(err) + } + s := New(Manifest{ID: "lastlogin-go", Title: "Last Login"}) + s.registrationInterval = 10 * time.Millisecond + s.cfg = config{ + port: strconv.Itoa(ln.Addr().(*net.TCPAddr).Port), + hostRegisterURL: srv.URL + "/api/solutions/register", + gatewayRegisterURL: srv.URL + solutionRegisterPath, + moduleRegisterURL: srv.URL + moduleRegisterPath, + moduleTokenURL: srv.URL + moduleRegistrationTokenPath, + solutionTokenURL: srv.URL + solutionRegistrationTokenPath, + internalToken: internalTokenTest, + solutionSecret: "s3cret", + } + ctx, cancel := context.WithCancel(context.Background()) + done := make(chan struct{}) + go func() { _ = s.serve(ctx, ln); close(done) }() + defer func() { cancel(); <-done }() + + // Two surfaces, each beating: six exchanges is at least two retries each. + waitFor(t, "exchange retries", func() bool { return exchanges.Load() >= 6 }) + out := buf.String() + if !strings.Contains(out, "host unavailable (status 502), retrying: accounts unreachable") { + t.Fatalf("an unavailable host was not reported as such:\n%s", out) + } + if strings.Contains(out, "SOLUTION_REGISTRATION_SECRETS") { + t.Fatalf("a 502 was reported as a provisioning fault:\n%s", out) + } +} diff --git a/solution_test.go b/solution_test.go index 91ab79d..2042079 100644 --- a/solution_test.go +++ b/solution_test.go @@ -133,14 +133,16 @@ func TestServeRegistersOnListenPort(t *testing.T) { base := "http://127.0.0.1:" + port waitFor(t, "health", func() bool { return getStatus(t, base+"/health") == http.StatusOK }) - // The advertised manifest port must equal the port the runtime actually binds. + // With no PUBLIC_URL the manifest is registered as a path on this backend — + // never on its own loopback listen address, which no browser but the + // developer's can reach — and that path is one this backend serves. manifestURL := frontendManifestURL(t, base) u, err := url.Parse(manifestURL) if err != nil { t.Fatalf("parse manifestUrl %q: %v", manifestURL, err) } - if u.Port() != port { - t.Errorf("manifestUrl port = %q, want listen port %q", u.Port(), port) + if u.IsAbs() || u.Host != "" || manifestURL != federationManifestPath { + t.Errorf("manifestUrl = %q, want the root-relative path %q", manifestURL, federationManifestPath) } if code := getStatus(t, base+u.Path); code != http.StatusOK { t.Errorf("GET manifestUrl = %d, want 200", code) @@ -542,6 +544,7 @@ func TestConfigValidate(t *testing.T) { moduleTokenURL: "http://gateway:42152/modules/_registration-token", solutionTokenURL: "http://gateway:42152/solutions/_registration-token", solutionSecret: "s3cret", + selfUpstream: "http://backend:8080", } if err := valid.validate(); err != nil { t.Fatalf("valid config rejected: %v", err) @@ -2497,3 +2500,29 @@ func (t internalTokenAuth) authorize(_ context.Context, req *http.Request) (time func (internalTokenAuth) invalidate() {} func (internalTokenAuth) succeeded() {} + +// TestManifestURLIsNeverTheListenAddress pins where the registered manifestUrl +// points. Unset PUBLIC_URL used to yield "http://localhost:/assets/...", +// this process's own loopback address, so every deployed solution registered a +// manifest no browser could load and the product showed it as failed to load. +// Without an explicit origin the URL is now the path on this backend, which the +// host resolves against the route by which it reaches the solution. +func TestManifestURLIsNeverTheListenAddress(t *testing.T) { + for _, tc := range []struct { + public string + want string + }{ + {public: "", want: "/assets/mf-manifest.json"}, + {public: "https://solutions.example.com/lastlogin", want: "https://solutions.example.com/lastlogin/assets/mf-manifest.json"}, + {public: "https://solutions.example.com/", want: "https://solutions.example.com/assets/mf-manifest.json"}, + } { + t.Setenv("PUBLIC_URL", tc.public) + t.Setenv("PORT", "8080") + s := New(Manifest{ID: "lastlogin-go"}) + s.cfg = loadConfig(context.Background(), s.manifest.ID) + frontend, _ := s.manifestMap()["frontend"].(map[string]any) + if got := frontend["manifestUrl"]; got != tc.want { + t.Errorf("PUBLIC_URL=%q: manifestUrl = %v, want %q", tc.public, got, tc.want) + } + } +}