Keep agent connector profiles safe across server upgrades

README.md

@@ -270,9 +270,21 @@ basecamp connect setup -P my-agent --serve 12345        # who may give it work,
 basecamp connect -P my-agent                            # the watcher
 ```
 
-Connect each agent once. Every connection rotates the agent's secret, so a
-second one, from this CLI or from the HTTP handshake on Basecamp's instructions
-page, disconnects the first and asks you to approve again.
+An approval may add a connection or replace an existing one: read the approval
+page before proceeding. Personal and team agents can have several connections
+where Basecamp enables adding them for that kind of agent. Otherwise a new
+approval replaces the existing connection. This applies to both the CLI and
+Basecamp's HTTP handshake. Do not reconnect just to upgrade the CLI.
+
+Each profile has its own projects, ledger, cursor, hold and local lock. When
+two connections serve the same project, a request both admit can be answered
+twice, so serve each project from one. Use a distinct `--device-name` for each
+profile on the same computer. Where Basecamp lists several connections, you
+can rename them and disconnect one without using Disconnect all. Reconnecting
+a profile on an add-capable server leaves its old connection live until you
+disconnect it; older credential files can remain live too. Do not delete them
+as part of setup. See the [upgrade and downgrade runbook](docs/agent-connections-rollout.md)
+for legacy state adoption and mixed-version precautions.
 
 In Claude Code, the `/basecamp-connect` skill runs the watcher and hands each
 request to a subagent. It ships in this CLI's plugin from 0.12.0 on, and an

docs/agent-connections-rollout.md

@@ -0,0 +1,119 @@
+# Agent connections: CLI-first rollout and rollback
+
+This CLI can merge and release before the server changes. It stores the
+connection secret as opaque data; no connection API, self-disconnect endpoint,
+or server feature switch is required. A second approval **may replace the
+existing connection**: it does on the current server, and for any agent whose
+kind (personal or team) doesn't have adding connections enabled. Where
+Basecamp has enabled it for the agent's kind, personal first and team later,
+the approval gains another independent connection instead. Consult
+the approval page; do not reconnect merely to upgrade the CLI.
+
+## Upgrade one host
+
+1. Stop **all older connectors for the same account/agent on this host**, normal
+   and shadow, including supervisors that could restart them. Keep credentials,
+   profiles and `connect.json` unchanged. Do not add a second profile yet.
+2. Start the existing profile with this build (`basecamp connect -P <profile>`).
+   After its credential proves the saved identity, the first verified run
+   renames `<account>-<agent>` to `<account>-<agent>-<profile>` under the old
+   instance lock. A WARN names both paths. The entire ledger moves: all filter
+   checkpoints, durable hold, held records, decisions and connection history.
+   A held connector stays held; migration never calls release.
+3. Check `basecamp connect status -P <profile>` and `basecamp connect doctor
+   -P <profile>`. Only if intended, use `basecamp connect release -P <profile>`;
+   release clears this profile's hold, not other profiles' holds. Held records
+   still require individual decisions. This does not promise replay: the existing
+   handoff age guard closes eligible requests from before a newly started run
+   as `discarded` / `before_this_run`, retaining their ledger rows. Requests
+   admitted during a hold are not necessarily individually held. Use a fresh
+   trusted event for handoff proof after a restart; inspect historical records
+   rather than assuming release will hand them off.
+4. For a legacy shadow, run `basecamp connect --shadow -P <profile>` first,
+   then stop it. Only then may `basecamp connect shadow promote -P <profile>`
+   move that profile's shadow ledger into normal state. Offline promotion
+   refuses the unverified legacy shadow instead of assigning it to a stale
+   profile. Normal state must not already have a ledger.
+5. After server support is confirmed, add other profiles with distinct device
+   names and project lists. Each profile has its own ledger, cursor and lock.
+   A second run of the same profile is refused. Deliberate project overlap is
+   allowed; requests admitted by both can be answered twice. There is no lease
+   or cross-profile claim.
+
+State is under `$XDG_STATE_HOME/basecamp/connect` and `connect-shadow` (default
+`~/.local/state/basecamp`). New profile directories canonicalize numeric account
+aliases so padded account ids cannot bypass the same-profile lock. Legacy
+lookup preserves the saved account spelling used by older builds; the adoption
+WARN names both paths. On downgrade, restore that saved spelling if padded.
+Before adoption, status, doctor and operator commands
+read the legacy folder without moving it. A working credential that reads back
+as a different person cannot adopt. The first verified profile owns the old
+state, not necessarily the first profile configured locally. A fresh sibling
+starts at the present unless `--since <event-id>` is supplied, and warns where
+older state remains. `--since` can replay work; inspect status before choosing
+it. If an existing profile already has its own folder, migration does not
+replace it or merge ledgers; inspect any remaining legacy state separately.
+
+The consumer namespace intentionally remains `basecamp-connect-<agent>` (and
+its shadow counterpart). Checkpoints are local rows in the **profile's isolated
+ledger**, not server-side consumers. Changing the namespace would orphan the
+adopted cursor; independent ledgers isolate identical keys.
+
+## Mixed builds are not a safe steady state
+
+After adoption an older build can recreate the old folder and run beside the
+new one. Its lock cannot see the new profile folder. Its fresh ledger would
+lose the old hold and cursor, and could dispatch duplicate work. Never restart
+an older supervisor while new connectors for that agent run on the same state
+root. Stop old builds before adoption; upgrade all launch commands. Distinct
+hosts/state roots remain independent, as before.
+
+## CLI downgrade (one chosen profile, offline)
+
+Stop every connector/supervisor for this account/agent in this state root. Back
+up the entire normal and shadow state directories with their permissions (they
+contain sensitive request data). Choose **one** profile for the old build;
+older builds cannot isolate several profiles for one agent. Ensure neither
+legacy destination exists. If one exists, stop: inspect and reconcile manually,
+never overwrite or merge SQLite ledgers.
+
+For each applicable group (`connect`, `connect-shadow`), rename the **whole**
+`<account>-<agent>-<chosen-profile>` directory back to `<account>-<agent>`.
+Do not move just `ledger.db`: journals and migration/promotion state belong to
+the same directory. Verify ownership/private permissions. Leave all other
+profile directories intact and stopped; do not delete them. Start only the
+chosen profile on the older binary and check status/doctor before releasing
+any hold. Downgrading without restoring the directory silently leaves held
+work/cursors behind and is unsafe. Preserve config/credentials unchanged.
+
+This is a CLI-state rollback, not a server-code rollback. Once own-secret
+connections exist, server code predating their authentication is unsupported.
+Server switch-off disables **adding** while keeping already-added connections
+live; use that switch for server rollback, not a casual code revert.
+
+## Credentials left outside the CLI
+
+Never read or delete a user's `~/.basecamp-agent/credentials.json` during setup.
+On an add-capable personal agent it can remain a live secret alongside this
+profile; reconnecting a profile can likewise leave its previous connection
+live. If the owner confirms an old credential is no longer used, have them
+disconnect **that connection** in Basecamp and then remove their unused file.
+Do not direct them to Disconnect all. Shared connections are managed in
+Adminland, personal ones on My > Agent. A lost one-shot handover can be cleaned
+up later by disconnecting the newest connection; its secret was not retained.
+
+## Provenance and proof boundaries
+
+Profile migration/operator fixes and tests reuse Rob Zolkos's
+[CLI #901](https://github.com/basecamp/basecamp-cli/pull/901) at `8ead7320`.
+Connection copy and lost-handover coverage reuse his
+[CLI #900](https://github.com/basecamp/basecamp-cli/pull/900) at `7ba7e962`, adapted
+for CLI-first release and Jeremy Daer's October 14 review. The per-connection
+logout/revoke guidance from Jeremy's [#851](https://github.com/basecamp/basecamp-cli/pull/851)
+is already in the base.
+
+Unit tests use synthetic fake servers and isolated local state. They prove
+identity-before-adoption, hold/cursor preservation, sibling isolation, lock
+refusal, operator fallback and safe copy—not real server lifecycle behavior.
+Real old/new CLI and server flag-off/on HTTP/OAuth/feed scenarios must be run
+in the separate isolated integration harness before enabling server adds.

internal/auth/agent.go

@@ -525,8 +525,8 @@ func (m *Manager) agentMintRefusal(resp *http.Response, body []byte, mint *agent
 }
 
 // ErrAgentCredentialRefused is the cause of a mint the token endpoint
-// refused for the credentials themselves: the agent was disconnected in
-// Basecamp, or connected on another computer, which replaced its secret.
+// refused for the credentials themselves: this connection may have been
+// disconnected in Basecamp or replaced by a newer approval.
 var ErrAgentCredentialRefused = errors.New("the agent's credential was refused")
 
 // clientRefusalCodes are the RFC 6749 §5.2 codes that say THE CREDENTIALS

internal/auth/agent_connect.go

@@ -314,12 +314,11 @@ func (m *Manager) ConnectAgent(ctx context.Context, opts AgentConnectOptions) (*
 // adoptAgentGrantOrSayWhatWasLost stores the handed-over credential, and
 // says what a failure costs.
 //
-// The poll that answered burned the one-time code and MINTED THE AGENT'S
-// SECRET — a rotation, for an agent something was already connected to —
-// so what this holds is the only live copy and the server will not hand it
-// over twice. A failure here (a refused mint, a profile that cannot be
-// written, a Ctrl-C landing in the gap) therefore leaves the agent
-// connected to nothing. Retrying this command starts a new ceremony, which
+// The poll that answered burned the one-time code and minted this connection's
+// secret, possibly replacing an existing connection. The server will not hand
+// it over twice. A failure here (a refused mint, a profile that cannot be
+// written, a Ctrl-C landing in the gap) leaves this connection without a kept
+// credential; other connections may still work. Retrying this command starts a new ceremony, which
 // is the right remedy; polling the spent code again is not, and neither is
 // looking for the secret somewhere.
 func (m *Manager) adoptAgentGrantOrSayWhatWasLost(ctx context.Context, disc *discovery, conn *AgentConnection, log func(string), opts ClientCredentialsOptions) (*LoginResult, error) {
@@ -344,14 +343,14 @@ func reconnectRemedy(err error) error {
 		return err
 	}
 	hinted := *e
-	hinted.Hint = "Disconnect the agent in Basecamp and run the connect command again; the secret this connection was handed is gone."
+	hinted.Hint = "Disconnect the newest connection in Basecamp (last in the list, where connections are listed), not Disconnect all, and run the connect command again; its secret is gone, so cleanup can wait."
 	return &hinted
 }
 
 // warnHandoverLost says what a failure after the handover costs, in the
 // one line an operator has to act on.
 func warnHandoverLost(log func(string)) {
-	log("warning: the connection was approved and the agent's client secret was handed over once, but keeping it failed. Disconnect the agent in Basecamp and connect again.")
+	log("warning: the connection was approved and its client secret was handed over once, but keeping it failed. Disconnect the newest connection in Basecamp (last in the list, where connections are listed), not Disconnect all, and connect again; its secret is gone, so cleanup can wait.")
 }
 
 // spentHandoverError marks an error raised once the poll had already been

internal/auth/agent_connect_test.go

@@ -387,7 +387,7 @@ func TestConnectAgentRefusesATokenWiderThanTheApproval(t *testing.T) {
 	require.Error(t, err)
 	assert.Contains(t, err.Error(), "wider than the scope")
 	assertNoAgentCredential(t, m)
-	assert.Contains(t, cl.joined(), "Disconnect the agent in Basecamp and connect again")
+	assertLostHandoverRemedy(t, cl.joined())
 	assert.NotContains(t, cl.joined(), "minted", "no token from a refused mint is kept")
 }
 
@@ -535,7 +535,7 @@ func TestConnectAgentRefusesAScopeItCannotStore(t *testing.T) {
 	assertNoAgentCredential(t, m)
 	// The poll was answered, so the code is burned and the secret minted
 	// whatever this process made of the answer.
-	assert.Contains(t, cl.joined(), "Disconnect the agent in Basecamp and connect again")
+	assertLostHandoverRemedy(t, cl.joined())
 	var e *output.Error
 	assert.ErrorAs(t, err, &e, "the refusal keeps its own class through the spent-handover marker")
 }
@@ -697,11 +697,9 @@ func TestConnectAgentStoresNothingWhenTheProfileCannotBeCommitted(t *testing.T)
 	require.Error(t, err)
 	assert.Contains(t, err.Error(), "bound to another account")
 	assertNoAgentCredential(t, m)
-	// The poll burned the code and rotated the agent's secret, so this
-	// failure leaves the agent connected to nothing. Saying "nothing was
-	// stored" alone would send the operator looking for a credential that
-	// no longer exists anywhere.
-	assert.Contains(t, cl.joined(), "Disconnect the agent in Basecamp and connect again")
+	// The poll burned the code and minted this connection's secret. Other
+	// connections may survive, but this one has no retained credential.
+	assertLostHandoverRemedy(t, cl.joined())
 }
 
 // TestConnectAgentRefusesTheLaunchpadFallback: Launchpad has no agent
@@ -842,7 +840,7 @@ func TestConnectAgentRefusesAHandoverWiderThanTheRequest(t *testing.T) {
 	assert.Contains(t, err.Error(), "read-only")
 	assert.Empty(t, as.calls(&as.tokenForms), "a handover wider than the request is never minted with")
 	assertNoAgentCredential(t, m)
-	assert.Contains(t, cl.joined(), "Disconnect the agent in Basecamp and connect again")
+	assertLostHandoverRemedy(t, cl.joined())
 }
 
 // TestConnectAgentHonorsASlowRetryAfter: a Retry-After longer than a
@@ -880,7 +878,7 @@ func TestConnectAgentHandsBackARemedyTheOperatorCanFollow(t *testing.T) {
 	require.Error(t, err)
 	var e *output.Error
 	require.ErrorAs(t, err, &e)
-	assert.Contains(t, e.Hint, "Disconnect the agent in Basecamp")
+	assertLostHandoverRemedy(t, e.Hint)
 	assert.NotContains(t, e.Hint, "--with-client-credentials")
 }
 
@@ -931,6 +929,14 @@ func TestConnectAgentReportsAnAccountLimitFromThePoll(t *testing.T) {
 // what it was going to do when it answered 200 — the code is burned and
 // the secret minted — so a body that never arrives costs exactly what a
 // refused mint costs and has to say the same thing.
+func assertLostHandoverRemedy(t *testing.T, text string) {
+	t.Helper()
+	assert.Contains(t, text, "Disconnect the newest connection in Basecamp")
+	assert.Contains(t, text, "last in the list, where connections are listed")
+	assert.Contains(t, text, "not Disconnect all")
+	assert.Contains(t, text, "cleanup can wait")
+}
+
 func TestConnectAgentTreatsAnInterruptedHandoverAsSpent(t *testing.T) {
 	as := startConnectAS(t)
 	as.pollHijack = true
@@ -939,7 +945,7 @@ func TestConnectAgentTreatsAnInterruptedHandoverAsSpent(t *testing.T) {
 	cl := &collectLogger{}
 	_, err := m.ConnectAgent(context.Background(), connectOptions(cl, newTestClock()))
 	require.Error(t, err)
-	assert.Contains(t, cl.joined(), "Disconnect the agent in Basecamp and connect again")
+	assertLostHandoverRemedy(t, cl.joined())
 	assertNoAgentCredential(t, m)
 	assert.Empty(t, as.calls(&as.tokenForms), "nothing was minted with a credential that never arrived")
 }

internal/commands/auth_agent.go

@@ -52,6 +52,17 @@ the agent's own OAuth client: this CLI mints an access token from it
 whenever it needs one, so there is no token to paste and none to renew by
 hand.
 
+An approval may add a connection or replace an existing one; read the
+approval page before proceeding. An agent, personal or team, keeps several
+connections where Basecamp has enabled adding them for its kind. Otherwise
+connecting replaces the existing connection, wherever it runs.
+
+The device name defaults to this computer's name. Give each profile its own
+--device-name when several on this computer connect the same agent.
+Reconnecting a profile stores its new credential in place of the old one.
+Where approval adds, the old connection stays live until disconnected in
+Basecamp; other credential files are not removed or revoked by this CLI.
+
 The account comes from the agent you approve — pass --account only to
 insist on one, and the connection is refused if the agent belongs to
 another.
@@ -65,8 +76,9 @@ Over SSH, in CI, or on a host with no display the browser is skipped and
 the link is yours to open on any device; --local forces a launch anyway,
 --no-browser skips it. Ctrl-C cancels the wait and stores nothing.
 
-Disconnecting is done in Basecamp: disconnecting the agent there kills the
-client secret this holds. Drop the local copy with ` + "`basecamp auth logout -P <profile>`" + `.`,
+Disconnecting is done in Basecamp: disconnecting this connection there kills
+the secret this holds. Where several connections are listed, choose this
+connection, not Disconnect all. Drop the local copy with ` + "`basecamp auth logout -P <profile>`" + `.`,
 		Annotations: map[string]string{AnnotationProfileMayCreate: "true"},
 		Args:        cobra.NoArgs,
 		RunE: func(cmd *cobra.Command, args []string) error {
@@ -94,7 +106,7 @@ client secret this holds. Drop the local copy with ` + "`basecamp auth logout -P
 		},
 	}
 
-	cmd.Flags().StringVar(&deviceName, "device-name", "", "Name this computer on the approval page (default: this host's name)")
+	cmd.Flags().StringVar(&deviceName, "device-name", "", "Name this device/connection (default: this host's name; use a distinct name for each profile)")
 	cmd.Flags().StringVar(&softwareName, "software-name", "", "Name the software this CLI runs inside, shown as \"Connected to\"")
 	cmd.Flags().StringVar(&scope, "scope", "", "OAuth scope to ask for: 'read' or 'full' (default full; the approval page can downgrade it)")
 	cmd.Flags().BoolVar(&noBrowser, "no-browser", false, "Print the link instead of opening a browser")

internal/commands/connect.go

@@ -57,9 +57,15 @@ line: events seen, admission's verdicts, and a "type":"request" line for each
 trusted request, which is the one to act on. Logs go to stderr. SIGINT and
 SIGTERM exit 130 and 143. A request made before the connector started is not
 picked up. --shadow admits and logs in an isolated state directory and hands
-off nothing. --hold sets a durable hold: intake and admission run, nothing is
-handed off, and earlier records wait for review, until basecamp connect
-release. Linux and macOS only.
+off nothing. --hold sets this profile's durable hold: intake and admission run,
+nothing is handed off, and earlier records wait for review, until basecamp
+connect release for this profile. Linux and macOS only.
+
+One connector runs per profile in this state directory. An agent can run
+several connections where Basecamp enables adding them for its kind; otherwise
+a new approval replaces the existing connection. Each profile
+serves its own projects and has its own hold. When two connections serve the
+same project, a request both admit can be answered twice.
 
   basecamp connect status             what it heard and holds
   basecamp connect doctor             what it needs to run
@@ -358,6 +364,10 @@ does not serve is handed off, and nobody is told. --watch-completions <project-i
 makes the agent hear every trusted completion in that project without being
 assigned.
 
+The list is this profile's own. When two connections of one agent serve the
+same project, a request both admit there can be answered twice, so serve
+each project from one of them.
+
 No directory is associated with a project here: the session handling the
 requests chooses the repo for each one.