Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -17,13 +17,13 @@ sentinels and defaults everything else to INTERNAL:

> ```go
> func errorResult(id string, err error) *compassv1internal.SessionsRequest {
> code := compassv1internal.RunnerErrorCode_RUNNER_ERROR_CODE_INTERNAL
> switch {
> case errors.Is(err, errAlreadyRunning):
> code = compassv1internal.RunnerErrorCode_RUNNER_ERROR_CODE_ALREADY_RUNNING
> case errors.Is(err, errSessionUnknown):
> code = compassv1internal.RunnerErrorCode_RUNNER_ERROR_CODE_NOT_FOUND
> }
> code := compassv1internal.RunnerErrorCode_RUNNER_ERROR_CODE_INTERNAL
> switch {
> case errors.Is(err, errAlreadyRunning):
> code = compassv1internal.RunnerErrorCode_RUNNER_ERROR_CODE_ALREADY_RUNNING
> case errors.Is(err, errSessionUnknown):
> code = compassv1internal.RunnerErrorCode_RUNNER_ERROR_CODE_NOT_FOUND
> }
> ```

and the Server's `runnerErrorToConnect`
Expand All @@ -33,11 +33,11 @@ and the Server's `runnerErrorToConnect`
> ```go
> switch e.GetCode() {
> case compassv1internal.RunnerErrorCode_RUNNER_ERROR_CODE_ALREADY_RUNNING:
> code = connect.CodeAlreadyExists
> code = connect.CodeAlreadyExists
> case compassv1internal.RunnerErrorCode_RUNNER_ERROR_CODE_NOT_FOUND:
> code = connect.CodeNotFound
> code = connect.CodeNotFound
> default:
> code = connect.CodeInternal
> code = connect.CodeInternal
> }
> ```

Expand All @@ -46,7 +46,7 @@ The canonical operator-fault path is the socket-path-too-long check, a bare

> ```go
> if len(path) > sunPathMax {
> return nil, fmt.Errorf("agent socket path %q is %d bytes, over the %d-byte AF_UNIX limit: shorten the Runner's --runtime-dir or the agent account id", path, len(path), sunPathMax)
> return nil, fmt.Errorf("agent socket path %q is %d bytes, over the %d-byte AF_UNIX limit: shorten the Runner's --runtime-dir or the agent account id", path, len(path), sunPathMax)
> }
> ```

Expand Down Expand Up @@ -190,7 +190,7 @@ returns `serveSocket`'s error directly (`host.go:172-175`), landing at

```go
case errors.Is(err, gateway.ErrOperatorConfig):
code = compassv1internal.RunnerErrorCode_RUNNER_ERROR_CODE_FAILED_PRECONDITION
code = compassv1internal.RunnerErrorCode_RUNNER_ERROR_CODE_FAILED_PRECONDITION
```

**Which socket.go paths qualify.** Classification rule: a failure is
Expand Down Expand Up @@ -267,11 +267,11 @@ before returning —

```go
func (d *dispatcher) errorResult(ctx context.Context, id string, err error) *compassv1internal.SessionsRequest {
// classify code as today, then log at a level chosen by class:
// context.Canceled / context.DeadlineExceeded -> skip (or Debug)
// INTERNAL -> Error
// classified operator/client-fault codes -> Warn
// build and return the result frame
// classify code as today, then log at a level chosen by class:
// context.Canceled / context.DeadlineExceeded -> skip (or Debug)
// INTERNAL -> Error
// classified operator/client-fault codes -> Warn
// build and return the result frame
}
```

Expand Down
162 changes: 162 additions & 0 deletions docs/operations/dogfood-provisioning.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,162 @@
# Dogfood door provisioning runbook

Bring a Compass **server + runner** pair up on a single host, over the
authenticated TLS network door, from the binaries this repository already
ships. No new tooling: cert generation, runner-token minting, and runner trust
are all existing `go/cmd/` binaries.

This is the loopback bring-up. Serving it onto a tailnet (a `tailscale serve`
front end) and baking the UI bundle are separate steps that consume the
`https://127.0.0.1:8443` door this runbook produces.

## Prerequisites

All of these before the first command — each is fail-fast, so a missing one
stops the bring-up at a named step rather than at the first RPC.

- **A Go toolchain on the serving host.** The four binaries below exist only as
`main` packages under `go/cmd/`; nothing in the tree installs them, and the
only build the repo carries (devenv's dev process) compiles `compass-server`
alone. On a fresh host the runbook's first real command is `command not
found` until you build them — which Shell 1 does. The pinned version is
`go = 1.26.5` (`.prototools`).
- **A reachable Postgres, and a DSN in the environment.** Both `compass-server`
and `compass-mint-runner-token` refuse to start without one, with the same
message — `a Postgres DSN is required: pass --database or set
$COMPASS_DATABASE_DSN` (`go/cmd/compass-server/main.go:160-163`;
`resolveDSN`, `go/cmd/compass-mint-runner-token/main.go:103-112`). The DSN is
not a formality: `store.Open` builds a pgx pool, forces a real connection
with `pool.Ping`, and applies the embedded migrations under an advisory lock
before the caller may serve (`go/internal/store/store.go:76-94`), so an
unreachable database fails the bring-up here rather than later. This
repository provisions no Postgres on a standalone host — a reachable
Postgres with a `compass` database is an explicit operator prerequisite.
- **The mint CLI and the server MUST resolve the SAME store.** The runner
token's SHA-256 hash is *written* by the mint path (`MintRunnerToken` →
`StoreRunnerTokenHash`, `go/internal/runnerhub/mint.go:103-115`) and
*resolved* by the server's runner door via `auth.ResolveToken` →
`st.ResolveTokenHash` (`go/internal/auth/token.go:102-116`); against two
different databases the runner's token is simply not found and enrollment
fails `Unauthenticated`. Exporting one `$COMPASS_DATABASE_DSN` for both
binaries is what guarantees this — `resolveDSN` "mirrors compass-server's
precedence exactly (flag wins, else `$COMPASS_DATABASE_DSN`, else error)"
(`go/cmd/compass-mint-runner-token/main.go:100-102`).

## Bring-up

Two shells: the server and the runner are both long-lived and hold the
foreground, so each gets its own. Shell 2 is a **new** shell and inherits
nothing from Shell 1 — re-export the same three literals before anything else.

The state dir is created and its mode pinned up front. `$STATE` holds the
admin-token file (0600, `go/server/network_door.go`), the socket (bound under
`umask 0177`, `go/server/socket.go`), and the runner token (0600), so nothing
inside it is world-readable — but the directory itself, left at the operator's
umask, is world-*traversable* (0755 under the default `umask 0022`). Pin it to
0700 so a private-by-name state dir is actually private. Use `mkdir -p` then
`chmod`, not `mkdir -m 700`: `-m` sets the mode on the leaf only (a missing
`~/.compass` ancestor lands 0755) and is a no-op on an existing dir, whereas
`chmod` is idempotent and pins the mode on every run.

```bash
# Shell 1 — prerequisites, then the server (which stays in the foreground).
export COMPASS_DATABASE_DSN=postgres://…/compass # see the DSN prerequisite
export STATE="$HOME/.compass/dogfood"

mkdir -p "$STATE" && chmod 700 "$STATE"

# Build all four binaries — nothing in the tree installs them (see above).
cd <repo>/go && go build -o "$STATE/bin/" \
./cmd/compass-gen-cert ./cmd/compass-server \
./cmd/compass-mint-runner-token ./cmd/compass-runner
export PATH="$STATE/bin:$PATH"

# One self-signed cert, its own CA: it is the server's --tls-cert AND the
# runner's --ca trust anchor. SANs default to 127.0.0.1,::1,localhost, and
# generation is skip-if-present so a restart never swaps the cert out from
# under a live server/runner pair (go/cmd/compass-gen-cert/main.go).
compass-gen-cert --cert-out "$STATE/tls.crt" --key-out "$STATE/tls.key"

# Long-lived: run() installs a SIGINT/SIGTERM signal context and server.Serve
# blocks on it, returning only on a termination signal. Leave this running — a
# literal copy-paste of the whole runbook into one shell never reaches the
# steps below. --socket is passed explicitly so the socket path is a known
# literal Shell 2 can reuse verbatim.
compass-server --listen 127.0.0.1:8443 \
--tls-cert "$STATE/tls.crt" --tls-key "$STATE/tls.key" --state-dir "$STATE" \
--socket "$STATE/server.sock"
```

```bash
# Shell 2 — a NEW shell, so it inherits nothing from Shell 1. Re-export the
# three values verbatim (same literals as above) before anything else.
export COMPASS_DATABASE_DSN=postgres://…/compass
export STATE="$HOME/.compass/dogfood"
export PATH="$STATE/bin:$PATH"

# Mint the runner's enrollment token (SubjectRunner keyspace), written 0600
# and idempotent against the store — not just against file presence. The admin
# account token does NOT work on the runner door.
compass-mint-runner-token --runner-id <id> --token-out "$STATE/runner.token"

# The UI needs no account id from this bring-up. It learns the caller's own
# account via the server's `WhoAmI` RPC right after the transport is up
# (`go/server/service.go:394`, server-derived and not admin-gated; the UI's
# `resolveCaller`, `apps/ui/src/live/client.ts:76`), so nothing bakes a caller
# id into the bundle and there is no `--admin-handle`-driven rebake. The only
# server-side value the UI build consumes is the admin bearer for
# `VITE_COMPASS_TOKEN` — the contents of the admin-token file `--listen` wrote
# (`$STATE/admin-token`, 0600), the same file the Verify step reads below.

# Also long-lived: the runner holds its Sessions bidi stream open for its life.
# The token is env only, never a flag (a flag leaks into the process table).
# --ca replaces the system root pool with exactly the generated cert.
COMPASS_RUNNER_TOKEN=$(cat "$STATE/runner.token") \
compass-runner --runner-id <id> --server https://127.0.0.1:8443 \
--ca "$STATE/tls.crt" --image <agent-image>
```

`compass-runner` fail-fasts without `--runner-id`/`$COMPASS_RUNNER_ID`,
`--server`/`$COMPASS_SERVER_ADDR`, `$COMPASS_RUNNER_TOKEN`, and
`--image`/`$COMPASS_AGENT_IMAGE` (`go/cmd/compass-runner/main.go:101-118`) — so
a missing one names itself rather than half-starting.

## Verify

From Shell 2, with the server still running. Do **not** use `grpcurl`: the
module registers no reflection service, so a grpcurl probe fails at descriptor
resolution before it reaches the auth interceptor — an unfalsifiable check.
Use `curl` against the connect handlers.

```bash
# 1. admin bearer, trusting the generated cert -> a GetServerInfo payload.
curl --cacert "$STATE/tls.crt" -X POST \
-H 'content-type: application/json' \
-H "authorization: Bearer $(cat "$STATE/admin-token")" \
-d '{}' https://127.0.0.1:8443/compass.v1.CompassService/GetServerInfo

# 2. same call, no authorization header -> connect code `unauthenticated`.
curl --cacert "$STATE/tls.crt" -X POST \
-H 'content-type: application/json' \
-d '{}' https://127.0.0.1:8443/compass.v1.CompassService/GetServerInfo
```

Plus the cross-door assertion: the `SubjectRunner` token enrolls over the
runner door (the `compass-runner` launch above succeeds and holds its stream)
while the admin account token is rejected there (`Unauthenticated`).

## Token lifetime (accepted risk)

`--listen` mints a fresh admin token on every start (`issueAndWriteAdminToken`
→ `auth.IssueAccountToken`, `go/internal/auth/token.go:41-65`, which INSERTs a
new hash and revokes nothing; `ResolveToken` accepts any persisted hash). Two
consequences:

- A baked UI bundle's bearer stays valid **indefinitely** across restarts — a
restart forces no rebake, and a rebake does not rotate the old bearer out.
- Every `--listen` restart accumulates an additional never-revoked live admin
token in the store. Rotating a leaked or superseded bundle's bearer needs an
explicit revoke; the store primitive exists and is tested
(`store.RevokeToken`, `go/internal/store/tokens.go:55-82`) but no operator
surface wires it yet (tracked as SEA-1490). That surface must land before the
bundle is served to anyone but its single operator.
Loading