Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -184,6 +184,9 @@ jobs:
- name: Test the skills sync
run: make test-sync-skills

- name: Test the skill drift check
run: make test-skill-drift

cli-surface:
name: CLI Surface Check
runs-on: ubuntu-latest
Expand Down
7 changes: 6 additions & 1 deletion .surface-skill-drift
Original file line number Diff line number Diff line change
@@ -1,6 +1,11 @@
# Known skill drift — skill references flags that don't exist on the surface command.
# Known skill drift — references the surface cannot confirm.
# These are pre-existing mismatches, not regressions. Fix in SKILL.md when convenient.
# Format: FLAG <command-path> <flag>
# CMD <path> — a command reference the check refuses, exactly as the DRIFT
# line's "baseline entry" names it: the deepest command that exists and
# the word past it, or the whole reference when nothing past "basecamp"
# is a command. This covers a hidden command (never in .surface) and a
# group that also runs bare (.surface cannot say it does).
FLAG basecamp comments list --in
FLAG basecamp comments update --in
FLAG basecamp people --in
Expand Down
31 changes: 27 additions & 4 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,14 +80,37 @@ session, stop and run it first.
*references* still exist in the `.surface` snapshot. It catches stale references, not
missing coverage, so adding a command breaks none of them.

Removal is only partly caught. `resolve_cmd` walks up to the nearest existing ancestor,
so dropping a nested subcommand leaves the reference resolving against its parent and the
check still passes — `basecamp setup <removed>` resolves as `basecamp setup`. Removing a
top-level command is caught; removing a subcommand is not. Don't lean on CI for this. Update the skill the change actually affects;
A reference has to resolve exactly: `basecamp setup <removed>` no longer passes by
falling back to `basecamp setup`. Words past a resolved command are allowed only where
they cannot name a subcommand — after a command that takes positional arguments, and
after a leaf that has none — so argument values and prose still pass; a word that runs
on into a filename or key (`upload report.pdf`, `config set project_id`) is read as the
argument it is. A word naming a command in `.surface-breaking`, the record of removals,
fails before either escape can take it, which is how removal is caught. Two limits
remain. Under a command that has both arguments and subcommands (`basecamp recordings`),
a subcommand that never existed passes as an argument value. And `.surface` cannot say
whether a group also runs bare (`basecamp skill` prints the skill file), so a word after
one fails even where the CLI would accept it; hidden commands are not in `.surface` at
all. Acknowledge a deliberate case with the baseline entry its DRIFT line names, in
`.surface-skill-drift`. `make test-skill-drift` runs the check against fixtures that hold
all of this.

Update the skill the change actually affects;
basecamp-doctor deliberately covers only doctor, setup and auth remediation;
basecamp-connect covers `basecamp auth agent connect` and `basecamp connect`, and its
evals run against its own SKILL.md (`make -C skill-evals eval-connect`).

**Hint commands**: a hint is an instruction an operator is about to run, so
`TestHintCommandsResolve` holds the commands hints name to the same exactness, against
the real command tree rather than `.surface`. It reads hint text in `internal/commands`
and `internal/connector/setup` — hints written inline, built into a variable, returned
by a helper, passed through a wrapper, or assigned to a hint-named field — and every
string in `connect.go`. Asking
the command lets it be exact where the script cannot: a word after a group passes only
if the group is not the root, runs, and its own argument validator accepts the word — which a subcommand
that never existed can still satisfy, under a group like `recordings` that takes one. It
checks commands, not flags.

```bash
bin/ci # The single command — run this
```
Expand Down
8 changes: 7 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -383,7 +383,7 @@ check-smoke-coverage: build

# Run all checks (local CI gate)
.PHONY: check
check: fmt-check vet lint lint-actions test test-e2e test-sync-skills check-naming check-surface check-skill-drift check-bare-groups check-lint-lockstep check-smoke-coverage provenance-check tidy-check
check: fmt-check vet lint lint-actions test test-e2e test-sync-skills check-naming check-surface check-skill-drift test-skill-drift check-bare-groups check-lint-lockstep check-smoke-coverage provenance-check tidy-check

# Lint GitHub Actions workflows (requires actionlint + zizmor)
.PHONY: lint-actions
Expand Down Expand Up @@ -459,6 +459,11 @@ check-skill-drift:
@scripts/check-skill-drift.sh skills/basecamp-doctor/SKILL.md
@scripts/check-skill-drift.sh skills/basecamp-connect/SKILL.md

# Verify the skill drift check itself: what it accepts, and what it refuses
.PHONY: test-skill-drift
test-skill-drift:
@scripts/test-skill-drift.sh

# Verify group commands show help bare (no RunE on parents with subcommands)
.PHONY: check-bare-groups
check-bare-groups:
Expand Down Expand Up @@ -622,6 +627,7 @@ help:
@echo " update-surface Regenerate committed .surface from the command tree"
@echo " check-surface-diff Compare CLI surface snapshots (fails on removals)"
@echo " check-skill-drift Verify skill references match CLI surface"
@echo " test-skill-drift Run the skill drift check against its own fixtures"
@echo ""
@echo "Dependencies:"
@echo " update-nix-hash Recompute Nix vendorHash via Docker"
Expand Down
48 changes: 46 additions & 2 deletions internal/commands/connect.go
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ import (
"fmt"
"net/http"
"os"
"path/filepath"
"runtime"
"strconv"
"strings"
Expand Down Expand Up @@ -862,8 +863,14 @@ func connectAccount(app *appctx.App, name string) (string, error) {
}
bound, err := canonicalAccount(p.AccountID)
if err != nil {
return "", output.ErrUsageHint(fmt.Sprintf("Profile %q is not bound to an account", name),
"Bind it: basecamp profile set "+shellQuote(name)+" account_id <id>")
// An entry that names an account nothing can use is not an unbound
// profile: say what is wrong with the account it does name.
if p.AccountID != "" {
return "", output.ErrUsageHint(
fmt.Sprintf("Profile %q names account %q, which is not an account ID", name, p.AccountID),
"Correct account_id in the config file that defines the profile.")
}
return "", unboundProfileError(name)
}
if accountGivenExplicitly(app) && !accountIDsEqual(app.Config.AccountID, bound) {
return "", output.ErrUsageHint(
Expand All @@ -873,6 +880,43 @@ func connectAccount(app *appctx.App, name string) (string, error) {
return bound, nil
}

// unboundProfileError refuses a profile that names no account, with the
// remedy that will work for this one.
//
// No command binds an account on its own — `profile` has none for it. The
// account is written by a command that stores a credential: `auth agent
// connect`, which takes the Agent's own account, and the headless logins
// (`auth login --with-token` or `--with-client-credentials` with --account).
// A bot user logged in through the browser, which binds nothing, so for a
// bot that already has its credential the account goes into its config entry
// by hand rather than through another login.
//
// Even `auth agent connect` refuses an accountless entry that is not the
// global config's, because the entry it would write stays shadowed by the
// system, repo or local config that defines it. For that profile the only
// remedy is the file the entry comes from.
func unboundProfileError(name string) error {
message := fmt.Sprintf("Profile %q is not bound to an account", name)
// The same question `auth agent connect` asks itself, so the hint is
// right whenever that command is: it takes an unbound global entry as
// its own to write. Where that reading is wrong — a trusted repo or
// local config defining the same accountless profile, which keeps
// shadowing the global entry after it is written — it is wrong for the
// command too, and telling them apart needs per-profile provenance the
// config layer does not keep.
//
// A global config that cannot be read is no reason to fail here: the
// command named below reports that itself, in full.
if unbound, err := globalProfileIsUnbound(name); err == nil && !unbound {
return output.ErrUsageHint(message,
"The entry is not the global config's, so no command can bind it. Add account_id to the config file that defines it.")
}
global := filepath.Join(config.GlobalConfigDir(), "config.json")
return output.ErrUsageHint(message,
"For an Agent, connecting it binds its own account: basecamp auth agent connect -P "+shellQuote(name)+
". A bot user's browser login binds none, so for a bot add account_id to the profile's entry in "+richtext.SanitizeSingleLine(global)+".")
}

// connectCredentialKind is the kind of credential the active profile holds,
// "" for none. A store that cannot be read is an error, not "none".
func connectCredentialKind(ctx context.Context, app *appctx.App) (string, error) {
Expand Down
79 changes: 79 additions & 0 deletions internal/commands/connect_unbound_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
//go:build unix

package commands

import (
"os"
"path/filepath"
"testing"

"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"

"github.com/basecamp/basecamp-cli/internal/appctx"
"github.com/basecamp/basecamp-cli/internal/config"
"github.com/basecamp/basecamp-cli/internal/output"
)

// writeGlobalConfig puts a config file where GlobalConfigDir will find it,
// and returns nothing: the test reads the hint, not the file.
func writeGlobalConfig(t *testing.T, contents string) {
t.Helper()
home := t.TempDir()
t.Setenv("XDG_CONFIG_HOME", home)
dir := filepath.Join(home, "basecamp")
require.NoError(t, os.MkdirAll(dir, 0o700))
require.NoError(t, os.WriteFile(filepath.Join(dir, "config.json"), []byte(contents), 0o600))
}

// hintOf is the hint an error carries for the operator to act on.
func hintOf(t *testing.T, err error) string {
t.Helper()
var e *output.Error
require.ErrorAs(t, err, &e)
return e.Hint
}

// No command binds an account on its own. Connecting an Agent writes its
// account; a bot user's browser login writes none — `auth login --account`
// binds only on the headless paths — so a bot's remedy is its config entry,
// and naming `auth login` would send the operator round the same loop.
func TestUnboundProfileHintNamesOnlyWhatBinds(t *testing.T) {
writeGlobalConfig(t, `{"profiles":{"agent":{"base_url":"https://3.basecampapi.com"}}}`)

hint := hintOf(t, unboundProfileError("agent"))

assert.Contains(t, hint, "basecamp auth agent connect -P agent")
assert.Contains(t, hint, "add account_id to the profile's entry in "+filepath.Join(config.GlobalConfigDir(), "config.json"))
assert.Contains(t, hint, "browser login binds none", "a headless login can bind, so the claim is about the browser")
assert.NotContains(t, hint, "auth login", "a browser login binds no account")
}

// Storing the credential again cannot bind a profile whose accountless entry
// comes from a system, repo or local config: both login paths refuse it,
// because the global entry they would write stays shadowed. Sending the
// operator there would be another command that fails on them.
func TestUnboundProfileHintNamesTheConfigFileWhenTheEntryIsNotGlobal(t *testing.T) {
writeGlobalConfig(t, `{"profiles":{"other":{"account_id":"999"}}}`)

hint := hintOf(t, unboundProfileError("agent"))

assert.Contains(t, hint, "account_id to the config file that defines it")
assert.NotContains(t, hint, "basecamp auth", "a command that would refuse this profile is no remedy")
}

// A profile that names an account nothing can parse is not an unbound one,
// and calling it unbound sends the operator to bind an account it already
// has. The account it does name is what is wrong.
func TestConnectAccountReportsAnUnusableAccountAsItself(t *testing.T) {
app := &appctx.App{Config: &config.Config{
Profiles: map[string]*config.ProfileConfig{"agent": {AccountID: "not-a-number"}},
}}

_, err := connectAccount(app, "agent")

require.Error(t, err)
assert.Contains(t, err.Error(), `Profile "agent" names account "not-a-number"`)
assert.NotContains(t, err.Error(), "not bound")
assert.Contains(t, hintOf(t, err), "Correct account_id")
}
Loading
Loading