diff --git a/docs/decisions/0003-smart-add-scope.md b/docs/decisions/0003-smart-add-scope.md new file mode 100644 index 0000000..f72463e --- /dev/null +++ b/docs/decisions/0003-smart-add-scope.md @@ -0,0 +1,129 @@ +# ADR-0003: Smart `add` Onboarding Scope (env / file / paste, optional AI-assisted parse) + +- **Status:** Proposed (awaiting CEO/LF sign-off) +- **Date:** 2026-07-08 +- **Owner:** CEO (LF) +- **Authority:** This memo governs the "smart api-key onboarding" scope (epic E13) only. Where it conflicts with `docs/decisions/0001-direction-lock.md` or `docs/decisions/0002-v1_1-scope.md`, this memo wins **for E13 only**. ADR-0001 and ADR-0002 remain the authority everywhere else. + +## Context + +`claudecm add` today is a declarative, flag-driven profile creator (E6-S3): every field +(`--base-url`, `--api-key`, `--model`, …) must be typed explicitly. That is script-perfect but +high-friction for the most common human task — "I just got a provider's onboarding blurb / an +`export` snippet / a config file, make me a profile from it." LF asked for three smarter entry +points: + +1. **From environment variables** — read the already-exported `ANTHROPIC_*` (and Codex) env vars + and materialize a profile. Zero network. +2. **From a config file** — point at a `.env`, shell `export` snippet, JSON/YAML/TOML config, or + another tool's config file and parse it into a profile draft. Zero network. +3. **From pasted free text** — paste an arbitrary blob (a provider's docs snippet, a chat log, a + half-formatted `export`), and have claudecm figure out `base_url` / `api_key` / `model`. + +Item 3 has two admissible strengths. A **local heuristic** pass (regex/structure extraction) needs +no network and is fully within the local-first lock. When the blob is too messy for heuristics, LF +wants an **opt-in AI-assisted** escalation whose defining constraint is: **the secret is stripped +locally first and never leaves the machine.** Only the desensitized (secret-free) text is sent to an +LLM, which returns a structured profile; the captured secret is re-injected locally. + +The AI escalation is the only part of E13 that makes an outbound network request, so it is the only +part that touches ADR-0001 Decision 8 ("no cloud") / the README's "never sees a request" language. +This ADR draws the line precisely. + +## Decision Summary + +Epic **E13 — Smart `add` Onboarding** adds three input adapters to `add`, layered so the +zero-network paths are the default and the network path is explicit opt-in: + +1. **`add --from-env`** — build a profile draft from the Claude Code / Codex environment-variable + allowlist. Local only. +2. **`add --from-file `** — auto-detect format (`.env` / shell `export` / JSON / YAML / TOML) + and parse into a profile draft. Local only. +3. **`add --from-text` / `--from-text -` (stdin)** — run the shared **local redaction + heuristic + extractor** over pasted text and produce a profile draft. Local only. +4. **`add --ai`** — escalation, **off by default**. Uses the shared extractor to strip and hold + secrets locally, sends only the desensitized text to an LLM, receives a structured profile, + re-injects the held secret locally, then routes through the normal `add` preview/validation. + This is the only E13 path that uses the network. + +All four paths converge on the existing `add` pipeline: they only produce a `config.Profile` draft, +which is then subject to the same `--dry-run`, redaction (NFR-S8), name validation (NFR-S5), +overwrite guard, and `SaveProfile` invariants. No E13 path writes a Claude Code or Codex tool file +directly, and no path auto-activates the new profile (activation stays `switch`). + +## Amendments to Prior Scope + +1. **ADR-0001 Decision 8 ("no cloud") is narrowly amended for E13's `--ai` path only.** claudecm + MAY make a single outbound LLM request to parse desensitized text into a profile draft, strictly + when the user opts in via `--ai`. This does not admit cloud sync, telemetry, a proxy, a gateway, + remote credential validation, or any always-on network behavior. Every non-`--ai` path, and the + default of every path, remains zero-network. +2. **README "never sees a request" is refined, not revoked.** The tool still never sees, proxies, or + routes a *tool* request (Claude Code / Codex traffic). The `--ai` path issues its own, + user-initiated, secret-free parse request and nothing else. Docs MUST state this distinction + plainly. +3. **No new command.** E13 adds flags/inputs to the existing `add` command (ADR-0001 Decision 3). + It does not introduce a new top-level command. + +## Locked Decisions + +1. **Secrets never transit the network.** The `--ai` path MUST run the local redaction pass first, + MUST hold captured secrets in-process, and MUST send only text from which secret-shaped tokens + have been removed/placeholdered. Re-injection of the real secret happens locally after the LLM + responds. A verifiable redaction step (secret token → placeholder) is an acceptance gate, not a + nicety. If redaction cannot be established for a candidate secret, that token is stripped + entirely rather than risk transit (no fallback that leaks). +2. **AI is opt-in and off by default.** Absent `--ai`, `add --from-text` uses only the local + heuristic and never touches the network. `--ai` must be typed explicitly per invocation; there is + no persisted "always use AI" mode in E13. +3. **LLM credential source.** `--ai` reuses the currently-active claudecm profile's endpoint and + key by default (dogfooding; no new secret to configure). `--ai-profile ` overrides which + profile's credentials are used for the parse call. The parse call never persists or logs the + credential it borrows. +4. **No remote validation.** No E13 path probes a provider endpoint to "verify" a key or model. + Drafts are produced from local + (optionally) LLM-structured data only; the user still confirms + via `--dry-run` / prompt before `SaveProfile`. +5. **No write-path bypass.** Every profile produced by E13 is saved through the same + `storage.SaveProfile` used by `add`/`import`, and any later activation still routes through + `internal/commit` + `internal/writepath`. E13 creates no side write path. +6. **Refuse, don't guess.** Consistent with NFR-S1 / CLAUDE.md no-fallback-writes: when a source + cannot be parsed into a valid draft (unreadable file, unrecognized format, LLM returns + non-conforming output), the command refuses with a clear error and writes nothing. It does not + silently emit a half-populated profile. + +## Explicit Non-Goals + +- No clipboard integration (paste is via argument or stdin `-`); OS clipboard access is deferred. +- No batch/multi-profile import from one blob; one draft per invocation. +- No provider auto-detection beyond field extraction (we do not map a base URL to an official + provider identity or claim support). +- No persisted AI configuration, no streaming UI, no model selection surface beyond the borrowed + profile's model. + +## Rationale + +The three entry points collapse the most common onboarding frictions into one command while keeping +claudecm's core promise intact: profiles stay explicit, local, inspectable YAML, and the user owns +the secret. Layering zero-network paths as the default with a clearly-fenced, secret-free, opt-in AI +escalation lets us say "smart" without becoming "a thing that phones home." The secret-never-transits +rule is what makes the AI path defensible against ADR-0001's spirit even while amending its letter. + +## Risks + +- **R1. Redaction miss → secret leak on the `--ai` path.** This is the load-bearing risk. + Mitigation: redaction is a tested gate with a conservative "strip on doubt" rule; `--ai` shows the + exact desensitized payload before sending when interactive; unit tests assert no secret-shaped + token survives into the outbound request. +- **R2. Heuristic false extraction** (wrong field grabbed). Mitigation: every path ends in + `--dry-run`/preview with redacted secrets; nothing is saved without confirmation. +- **R3. Scope creep toward "AI everywhere."** Mitigation: `--ai` is per-invocation, single-request, + parse-only, no persistence; this ADR forbids always-on modes. +- **R4. Borrowed-credential surprise** (user didn't expect their active key to be used for a parse + call). Mitigation: interactive confirmation names the profile whose credentials will be used + before the call; `--ai-profile` makes it explicit. + +## Supersedes + +For E13 only: ADR-0001 Decision 8's blanket no-network stance as applied to the single opt-in +`--ai` parse request, and the README's "never sees a request" phrasing as applied to that same +user-initiated, secret-free call. Everything else in ADR-0001 and ADR-0002 stands. diff --git a/docs/plan/epics.md b/docs/plan/epics.md index 572ca29..4dfea0f 100644 --- a/docs/plan/epics.md +++ b/docs/plan/epics.md @@ -203,6 +203,26 @@ No story silently expands v1 scope: no MCP, no cloud, no GUI, no Gemini CLI / Cu --- +## E13. Smart `add` Onboarding + +> Authority: ADR-0003 (`docs/decisions/0003-smart-add-scope.md`) governs this epic. It narrowly amends ADR-0001 Decision 8 for the opt-in `--ai` parse path only; everything else stays zero-network and local-first. + +**Goal.** Collapse the three highest-friction onboarding flows into `add`: build a profile draft from environment variables, from a config file, or from pasted free text. Pasted text is parsed by a local redaction+heuristic extractor by default; an explicit `--ai` escalation strips secrets locally, sends only desensitized text to an LLM (borrowing the active profile's credentials), and re-injects the secret locally. Every path ends in the existing `add` preview/validation/save pipeline and never auto-activates or writes a tool file directly. + +**Acceptance criteria.** +- `add --from-env` materializes a draft from the Claude Code / Codex env-var allowlist; zero network; missing required fields (no key) refused with a clear message. +- `add --from-file ` auto-detects `.env` / shell `export` / JSON / YAML / TOML and parses into a draft; unreadable/unrecognized input is refused (NFR-S1), never half-populated; zero network. +- A local redaction+heuristic extractor library exists (`internal/blobparse` or equivalent): given arbitrary text it returns extracted core fields **plus** a desensitized copy in which no secret-shaped token survives; "strip on doubt" — a candidate secret that cannot be safely placeholdered is removed, never passed through. +- `add --from-text ` / `add --from-text -` (stdin) runs the local extractor and produces a draft; zero network. +- `add --ai` is off by default and requires the flag per invocation. It runs the extractor, holds secrets locally, sends only the desensitized text to the LLM (active profile by default, `--ai-profile ` override), maps the structured response into the profile schema, re-injects the held secret locally, and refuses if the LLM output does not conform. Interactive runs name the credential-lending profile and show the desensitized payload before the call. +- No E13 path probes a remote provider to validate a key/model; no secret is logged or persisted by the parse call; all drafts save through `storage.SaveProfile` and any activation still routes through `internal/commit` + `internal/writepath`. +- Secrets are redacted by default in every preview (`--dry-run` / prompt) per NFR-S8; profile-name validation (NFR-S5) and overwrite guard are unchanged. +- Docs and `--help` state the local-first default and that only the opt-in `--ai` path makes a (secret-free) network request. + +**Stories.** E13-S1, E13-S2, E13-S3, E13-S4, E13-S5. + +--- + ## Story count - E1: 7 stories @@ -217,4 +237,5 @@ No story silently expands v1 scope: no MCP, no cloud, no GUI, no Gemini CLI / Cu - E10: 4 stories - E11: 4 stories - E12: 5 stories -- **Total: 12 epics, 69 stories.** +- E13: 5 stories +- **Total: 13 epics, 74 stories.** diff --git a/docs/plan/stories/E13-S1.md b/docs/plan/stories/E13-S1.md new file mode 100644 index 0000000..bf5f418 --- /dev/null +++ b/docs/plan/stories/E13-S1.md @@ -0,0 +1,35 @@ +# E13-S1 — Local redaction + heuristic blob extractor + +**User story.** +As a claudecm user pasting a messy blob (provider onboarding text, an `export` snippet, a chat log), I want claudecm to locally detect the base URL / API key / model and produce both the extracted fields and a secret-free desensitized copy of my text, so later stories can build a profile draft without any secret ever needing to leave my machine. + +**Acceptance criteria.** +- A new package (`internal/blobparse`) exposes a pure function that takes arbitrary text and returns: extracted core fields (`base_url`, `api_key`, `model`, `small_fast_model`, `provider` where inferable), a map of captured secrets, and a **desensitized** copy of the input. +- Extraction is heuristic and local: recognizes `KEY=VALUE`, shell `export KEY=VALUE`, JSON/YAML `"key": "value"`, and common label forms (`API Key: ...`, `Base URL: ...`) for the Anthropic/Codex field vocabulary. +- Secret detection covers `ANTHROPIC_API_KEY`, `ANTHROPIC_AUTH_TOKEN`, `OPENAI_API_KEY`, and generic `sk-`/token-shaped values; every detected secret is replaced by a stable placeholder in the desensitized copy. +- **Strip-on-doubt:** a token that looks secret-shaped but cannot be safely mapped to a placeholder is removed from the desensitized copy entirely rather than passed through. No secret-shaped token survives desensitization. +- Pure function, no I/O, no network, deterministic; provider inference (if any) never claims official support. + +**PRD/architecture refs.** ADR-0003 Locked Decisions 1 & 6; PRD FR-1, NFR-S1, NFR-S8; Architecture §2.1. + +**Test plan.** +- Happy: an `export ANTHROPIC_BASE_URL=... ANTHROPIC_AUTH_TOKEN=sk-...` blob → base_url + key extracted; desensitized copy contains a placeholder, not the key. +- Happy: a prose blob (`Base URL: https://... , API Key: sk-...`) → fields extracted. +- Edge: blob with a bare `sk-xxxx` and no label → captured as a secret; desensitized copy has no `sk-` token. +- Edge: blob with no recognizable fields → empty extraction, no panic, desensitized copy equals input (nothing secret-shaped to strip). +- Property: for a corpus of blobs, assert no secret-shaped token appears in any desensitized output. + +**No fallback writes.** N/A — pure library; no disk writes. + +**Complexity.** M. + +**Dependencies.** E1-S1, E6-S3. + +**Ready-for-dev checklist.** +- [x] PRD/architecture refs correct. +- [x] AC testable. +- [x] Happy + edge test rows. +- [x] No-fallback reminder present. +- [x] ADR-0003 scope only. +- [x] Deps satisfied. +- [x] Complexity set. diff --git a/docs/plan/stories/E13-S2.md b/docs/plan/stories/E13-S2.md new file mode 100644 index 0000000..8b1d788 --- /dev/null +++ b/docs/plan/stories/E13-S2.md @@ -0,0 +1,33 @@ +# E13-S2 — `add --from-env` + +**User story.** +As a user who already has `ANTHROPIC_*` (or Codex) variables exported in my shell, I want `claudecm add --from-env` to read them and build a profile draft, so I don't retype values that are already in my environment. + +**Acceptance criteria.** +- `add --from-env` reads the Claude Code / Codex environment-variable allowlist (`ANTHROPIC_BASE_URL`, `ANTHROPIC_AUTH_TOKEN`/`ANTHROPIC_API_KEY`, `ANTHROPIC_MODEL`, `ANTHROPIC_SMALL_FAST_MODEL`, and the Codex equivalents) and populates the corresponding profile core fields. +- Env reads route through `internal/envextract` (not ad-hoc `os.Getenv` scattered in `cmd/`). +- Explicit flags win over env: e.g. `--model` overrides `ANTHROPIC_MODEL` from the environment. +- If no API key/token is present in the environment, the command refuses with a clear message rather than writing a keyless profile (NFR-S1, no fallback). +- Reuses the existing `add` pipeline: `--dry-run`, redaction, name validation, overwrite guard, `SaveProfile`; does not auto-activate. Zero network. + +**PRD/architecture refs.** ADR-0003 Decision Summary item 1; ADR-0001 Decision 4 (secondary env flow); PRD FR-1, NFR-E1, NFR-S1; Architecture §2.1. + +**Test plan.** +- Happy: env has base_url + auth token + model → draft populated; `--dry-run` shows redacted key. +- Edge: env has base_url + model but no key → refused with "no API key found in environment". +- Edge: `--from-env` plus explicit `--model foo` → `foo` wins over env model. + +**No fallback writes.** No keyless/partial profile is written; refuse when required fields are absent. + +**Complexity.** S. + +**Dependencies.** E6-S3, E5-S1 (envextract allowlist). + +**Ready-for-dev checklist.** +- [x] PRD/architecture refs correct. +- [x] AC testable. +- [x] Happy + edge test rows. +- [x] No-fallback reminder present. +- [x] ADR-0003 scope only. +- [x] Deps satisfied. +- [x] Complexity set. diff --git a/docs/plan/stories/E13-S3.md b/docs/plan/stories/E13-S3.md new file mode 100644 index 0000000..8048855 --- /dev/null +++ b/docs/plan/stories/E13-S3.md @@ -0,0 +1,34 @@ +# E13-S3 — `add --from-file` + +**User story.** +As a user with a provider's credentials sitting in a `.env`, a shell `export` snippet, or a JSON/YAML/TOML config, I want `claudecm add --from-file ` to parse that file into a profile draft, so onboarding is one command instead of hand-copying fields. + +**Acceptance criteria.** +- `add --from-file ` auto-detects the format by content/extension: dotenv, shell `export` lines, JSON, YAML, TOML. +- Recognized field vocabulary maps to profile core fields (base_url, api_key/auth_token, model, small_fast_model, provider where present). +- Explicit flags override values parsed from the file. +- Unreadable file, or a file whose content matches no recognized format / yields no usable fields, is refused with a clear message; no partial profile is written (NFR-S1). +- File is read read-only; nothing outside `add`'s normal write to `~/.claudecm/profiles/.yaml` is touched. Zero network. Reuses `add` `--dry-run`/redaction/validation/overwrite/save; no auto-activation. + +**PRD/architecture refs.** ADR-0003 Decision Summary item 2, Locked Decision 6; PRD FR-1, NFR-S1, NFR-S8; Architecture §2.1. + +**Test plan.** +- Happy: a `.env` with `ANTHROPIC_BASE_URL=` + `ANTHROPIC_AUTH_TOKEN=` → draft populated; dry-run redacts key. +- Happy: a JSON `{ "base_url": ..., "api_key": ..., "model": ... }` → draft populated. +- Edge: a shell `export ANTHROPIC_MODEL=...` snippet → model parsed. +- Edge: nonexistent path → refused "cannot read file"; a binary/garbage file → refused "unrecognized config format". + +**No fallback writes.** Unrecognized/unreadable input refuses; never emit a half-populated profile. + +**Complexity.** M. + +**Dependencies.** E6-S3, E1-S1. + +**Ready-for-dev checklist.** +- [x] PRD/architecture refs correct. +- [x] AC testable. +- [x] Happy + edge test rows. +- [x] No-fallback reminder present. +- [x] ADR-0003 scope only. +- [x] Deps satisfied. +- [x] Complexity set. diff --git a/docs/plan/stories/E13-S4.md b/docs/plan/stories/E13-S4.md new file mode 100644 index 0000000..d3a155b --- /dev/null +++ b/docs/plan/stories/E13-S4.md @@ -0,0 +1,34 @@ +# E13-S4 — `add --from-text` (local heuristic paste) + +**User story.** +As a user who just copied a messy blob (provider docs, a chat message, a half-formatted snippet), I want `claudecm add --from-text ""` (or `--from-text -` to read stdin) to locally extract the fields into a profile draft, so I can onboard without hand-parsing and without anything leaving my machine. + +**Acceptance criteria.** +- `add --from-text ` and `add --from-text -` (stdin) run the `internal/blobparse` (E13-S1) local extractor and populate the profile draft from the extracted fields. +- Zero network on this path; it is the local-heuristic default of the paste flow. +- Explicit flags override extracted values. +- When the extractor finds no API key/token, the command refuses with a clear message (NFR-S1) unless a key is supplied by flag; no keyless profile is written. +- Reuses `add` `--dry-run`/redaction/validation/overwrite/save; secrets redacted in preview; no auto-activation. + +**PRD/architecture refs.** ADR-0003 Decision Summary item 3, Locked Decisions 1, 2, 6; PRD FR-1, NFR-S1, NFR-S8. + +**Test plan.** +- Happy: `--from-text "export ANTHROPIC_BASE_URL=... ANTHROPIC_AUTH_TOKEN=sk-..."` → draft populated; dry-run redacts key. +- Happy: `--from-text -` reading a prose blob from stdin → fields extracted. +- Edge: blob with base_url + model but no key → refused unless `--api-key` supplied. +- Edge: blob with nothing recognizable → refused "could not extract profile fields from text". + +**No fallback writes.** No keyless/partial profile is written; refuse when the key is absent and unsupplied. + +**Complexity.** M. + +**Dependencies.** E13-S1, E6-S3. + +**Ready-for-dev checklist.** +- [x] PRD/architecture refs correct. +- [x] AC testable. +- [x] Happy + edge test rows. +- [x] No-fallback reminder present. +- [x] ADR-0003 scope only. +- [x] Deps satisfied. +- [x] Complexity set. diff --git a/docs/plan/stories/E13-S5.md b/docs/plan/stories/E13-S5.md new file mode 100644 index 0000000..d0452b2 --- /dev/null +++ b/docs/plan/stories/E13-S5.md @@ -0,0 +1,37 @@ +# E13-S5 — `add --ai` (opt-in, secret-free LLM-assisted parse) + +**User story.** +As a user whose pasted blob is too messy for the local heuristic, I want `claudecm add --from-text --ai` to strip my secrets locally, send only the desensitized text to an LLM using my own already-configured credentials, get back a structured profile, and re-inject the secret locally — so I get smart parsing without any secret ever leaving my machine. + +**Acceptance criteria.** +- `--ai` is off by default and must be passed explicitly per invocation; there is no persisted "always AI" mode. +- The path runs `internal/blobparse` (E13-S1) first, holds captured secrets in-process, and constructs an outbound request containing **only** the desensitized text — asserted by test to contain no secret-shaped token. +- Credentials for the parse call default to the currently-active claudecm profile's `base_url`/`api_key`; `--ai-profile ` overrides which profile lends credentials. The borrowed credential is never logged or persisted. +- The LLM is asked to return a structured profile (JSON conforming to the core-field schema). Non-conforming output is refused (NFR-S1); no guessed/half profile is written. +- After a conforming response, the held secret is re-injected locally into the draft, then the normal `add` preview/validation/overwrite/save runs; secrets redacted in preview; no auto-activation. +- Interactive runs name the credential-lending profile and display the exact desensitized payload before sending; non-interactive runs proceed only with `--ai` explicitly set. +- No remote provider probe/validation of the resulting key or model. + +**PRD/architecture refs.** ADR-0003 Amendments 1-2, Locked Decisions 1-4; PRD FR-1, NFR-S1, NFR-S8; ADR-0001 Decision 8 (as amended by ADR-0003 for this path only). + +**Test plan.** +- Happy (mocked LLM): desensitized blob → mock returns conforming JSON → draft built, secret re-injected locally, dry-run redacts key. +- Security: assert the outbound request body contains no secret-shaped token from the input (redaction gate). +- Edge: mock returns malformed/non-conforming output → refused, nothing written. +- Edge: no active profile and no `--ai-profile` → refused "no credentials available for --ai parse". +- Edge: `--ai-profile missing` → refused with unknown-profile error. + +**No fallback writes.** Non-conforming LLM output or missing credentials refuse; never write a guessed profile. + +**Complexity.** L. + +**Dependencies.** E13-S1, E13-S4, E6-S3, E5 (resolver/active-profile lookup). + +**Ready-for-dev checklist.** +- [x] PRD/architecture refs correct. +- [x] AC testable. +- [x] Happy + edge + security test rows. +- [x] No-fallback reminder present. +- [x] ADR-0003 scope only (network amendment gated to this path). +- [x] Deps satisfied. +- [x] Complexity set. diff --git a/internal/blobparse/parse.go b/internal/blobparse/parse.go new file mode 100644 index 0000000..45f30dd --- /dev/null +++ b/internal/blobparse/parse.go @@ -0,0 +1,404 @@ +// Package blobparse extracts claudecm profile fields from arbitrary pasted +// text while producing a secret-free copy of the same text. +package blobparse + +import ( + "fmt" + "regexp" + "sort" + "strconv" + "strings" + + "github.com/a2d2-dev/claudecm/internal/config" + "github.com/a2d2-dev/claudecm/internal/envextract" +) + +// Result is the deterministic output of Parse. +// +// CapturedSecrets maps each placeholder present in Desensitized back to the +// original secret value. Callers that later send Desensitized outside the +// process can keep this map local and re-inject only after the remote parse +// result returns. +type Result struct { + Core config.CoreConfig + CapturedSecrets map[string]string + Desensitized string +} + +// Parse extracts profile core fields from text and returns a desensitized copy. +// +// It performs no I/O, no network access, and uses only deterministic local +// heuristics. +func Parse(text string) Result { + registry := newSecretRegistry() + candidates := extractCandidates(text) + core := buildCore(candidates) + + for _, candidate := range candidates { + if candidate.kind == fieldAPIKey && candidate.value != "" { + registry.placeholderFor(candidate.value) + } + } + registerGenericSecrets(text, candidates, registry) + + desensitized := registry.desensitize(text) + desensitized = scrubResidualSecretShapes(desensitized, candidates) + + return Result{ + Core: core, + CapturedSecrets: registry.captured, + Desensitized: desensitized, + } +} + +type fieldKind int + +const ( + fieldBaseURL fieldKind = iota + fieldAPIKey + fieldModel + fieldSmallFastModel + fieldProvider +) + +type fieldAlias struct { + kind fieldKind + pattern string + priority int + providerHint string +} + +type fieldCandidate struct { + kind fieldKind + value string + priority int + position int + providerHint string +} + +type selectedCandidate struct { + value string + priority int + position int + providerHint string + set bool +} + +func extractCandidates(text string) []fieldCandidate { + valuePattern := `("(?:\\.|[^"\\])*"|'[^'\n]*'|` + "`" + `[^` + "`" + `\n]*` + "`" + `|[^\s,;#}\]]+)` + candidates := []fieldCandidate{} + for _, alias := range fieldAliases() { + pattern := `(?i)(^|[\s{[,;])(?:export[ \t]+)?["']?` + + alias.pattern + `["']?[ \t]*[:=][ \t]*` + valuePattern + re := regexp.MustCompile(pattern) + matches := re.FindAllStringSubmatchIndex(text, -1) + for _, match := range matches { + if len(match) < 6 || match[4] < 0 || match[5] < 0 { + continue + } + value := cleanValue(text[match[4]:match[5]], alias.kind) + if value == "" { + continue + } + candidates = append(candidates, fieldCandidate{ + kind: alias.kind, + value: value, + priority: alias.priority, + position: match[4], + providerHint: alias.providerHint, + }) + } + } + sort.SliceStable(candidates, func(i, j int) bool { + return candidates[i].position < candidates[j].position + }) + return candidates +} + +func fieldAliases() []fieldAlias { + return []fieldAlias{ + {kind: fieldBaseURL, pattern: exactAlias(envextract.EnvBaseURL), priority: 0, providerHint: "anthropic"}, + {kind: fieldBaseURL, pattern: exactAlias("OPENAI_BASE_URL"), priority: 1, providerHint: "openai-compatible"}, + {kind: fieldBaseURL, pattern: aliasWords("base", "url"), priority: 3}, + + {kind: fieldAPIKey, pattern: exactAlias(envextract.EnvAuthToken), priority: 0, providerHint: "anthropic"}, + {kind: fieldAPIKey, pattern: exactAlias("ANTHROPIC_API_KEY"), priority: 1, providerHint: "anthropic"}, + {kind: fieldAPIKey, pattern: exactAlias("OPENAI_API_KEY"), priority: 1, providerHint: "openai-compatible"}, + {kind: fieldAPIKey, pattern: aliasWords("auth", "token"), priority: 3}, + {kind: fieldAPIKey, pattern: aliasWords("api", "key"), priority: 4}, + {kind: fieldAPIKey, pattern: aliasWords("access", "token"), priority: 5}, + + {kind: fieldModel, pattern: exactAlias(envextract.EnvModel), priority: 0, providerHint: "anthropic"}, + {kind: fieldModel, pattern: exactAlias("CODEX_MODEL"), priority: 1, providerHint: "openai-compatible"}, + {kind: fieldModel, pattern: aliasWords("model"), priority: 3}, + + {kind: fieldSmallFastModel, pattern: exactAlias(envextract.EnvSmallFastModel), priority: 0, providerHint: "anthropic"}, + {kind: fieldSmallFastModel, pattern: aliasWords("small", "fast", "model"), priority: 2}, + + {kind: fieldProvider, pattern: exactAlias("CODEX_MODEL_PROVIDER"), priority: 0}, + {kind: fieldProvider, pattern: aliasWords("model", "provider"), priority: 1}, + {kind: fieldProvider, pattern: aliasWords("provider"), priority: 2}, + } +} + +func exactAlias(name string) string { + return regexp.QuoteMeta(name) +} + +func aliasWords(words ...string) string { + parts := make([]string, 0, len(words)) + for _, word := range words { + parts = append(parts, regexp.QuoteMeta(word)) + } + return strings.Join(parts, `[\s_-]*`) +} + +func buildCore(candidates []fieldCandidate) config.CoreConfig { + selected := map[fieldKind]selectedCandidate{} + for _, candidate := range candidates { + current := selected[candidate.kind] + if !current.set || + candidate.priority < current.priority || + (candidate.priority == current.priority && candidate.position < current.position) { + selected[candidate.kind] = selectedCandidate{ + value: candidate.value, + priority: candidate.priority, + position: candidate.position, + providerHint: candidate.providerHint, + set: true, + } + } + } + + core := config.CoreConfig{} + if c := selected[fieldBaseURL]; c.set { + core.BaseURL = c.value + } + if c := selected[fieldAPIKey]; c.set { + core.APIKey = c.value + } + if c := selected[fieldModel]; c.set { + core.Model = c.value + } + if c := selected[fieldSmallFastModel]; c.set { + core.SmallFastModel = c.value + } + if c := selected[fieldProvider]; c.set { + core.Provider = c.value + } else { + core.Provider = inferProvider(selected) + } + return core +} + +func inferProvider(selected map[fieldKind]selectedCandidate) string { + if c := selected[fieldAPIKey]; c.set && c.providerHint != "" { + return c.providerHint + } + if c := selected[fieldBaseURL]; c.set && c.providerHint != "" { + return c.providerHint + } + if c := selected[fieldModel]; c.set && c.providerHint != "" { + return c.providerHint + } + return "" +} + +func cleanValue(raw string, kind fieldKind) string { + value := strings.TrimSpace(raw) + if value == "" { + return "" + } + + switch { + case strings.HasPrefix(value, `"`) && strings.HasSuffix(value, `"`): + if unquoted, err := strconv.Unquote(value); err == nil { + return unquoted + } + return strings.Trim(value, `"`) + case strings.HasPrefix(value, "'") && strings.HasSuffix(value, "'"): + return strings.Trim(value, "'") + case strings.HasPrefix(value, "`") && strings.HasSuffix(value, "`"): + return strings.Trim(value, "`") + default: + if kind == fieldAPIKey { + return value + } + return strings.TrimRight(value, ".,)]}") + } +} + +type secretRegistry struct { + bySecret map[string]string + captured map[string]string + next int +} + +func newSecretRegistry() *secretRegistry { + return &secretRegistry{ + bySecret: map[string]string{}, + captured: map[string]string{}, + } +} + +func (r *secretRegistry) placeholderFor(secret string) string { + secret = strings.TrimSpace(secret) + if secret == "" { + return "" + } + if placeholder, ok := r.bySecret[secret]; ok { + return placeholder + } + r.next++ + placeholder := fmt.Sprintf("{{CLAUDECM_SECRET_%d}}", r.next) + r.bySecret[secret] = placeholder + r.captured[placeholder] = secret + return placeholder +} + +func (r *secretRegistry) placeholderForNormalized(secret string) string { + return r.placeholderFor(normalizeSecretToken(secret)) +} + +func (r *secretRegistry) desensitize(text string) string { + secrets := make([]string, 0, len(r.bySecret)) + for secret := range r.bySecret { + secrets = append(secrets, secret) + } + sort.Slice(secrets, func(i, j int) bool { + if len(secrets[i]) == len(secrets[j]) { + return secrets[i] < secrets[j] + } + return len(secrets[i]) > len(secrets[j]) + }) + + desensitized := text + for _, secret := range secrets { + desensitized = strings.ReplaceAll(desensitized, secret, r.bySecret[secret]) + } + return desensitized +} + +func registerGenericSecrets(text string, candidates []fieldCandidate, registry *secretRegistry) { + for _, corePattern := range secretShapePatterns() { + re := regexp.MustCompile(`(^|[^A-Za-z0-9_-])(` + corePattern + `)`) + matches := re.FindAllStringSubmatchIndex(text, -1) + for _, match := range matches { + if len(match) < 6 || match[4] < 0 || match[5] < 0 { + continue + } + registry.placeholderForNormalized(text[match[4]:match[5]]) + } + } + registerHighEntropySecrets(text, candidates, registry) +} + +func scrubResidualSecretShapes(text string, candidates []fieldCandidate) string { + desensitized := text + for _, corePattern := range secretShapePatterns() { + re := regexp.MustCompile(`(^|[^A-Za-z0-9_-])(` + corePattern + `)`) + desensitized = re.ReplaceAllString(desensitized, `${1}`) + } + return scrubResidualHighEntropyTokens(desensitized, candidates) +} + +func registerHighEntropySecrets(text string, candidates []fieldCandidate, registry *secretRegistry) { + for _, rawToken := range strings.Fields(text) { + token := highEntropyTokenCandidate(rawToken) + if isHighEntropySecretToken(token, candidates) { + registry.placeholderFor(token) + } + } +} + +func scrubResidualHighEntropyTokens(text string, candidates []fieldCandidate) string { + fields := strings.Fields(text) + if len(fields) == 0 { + return text + } + + desensitized := text + for _, rawToken := range fields { + token := highEntropyTokenCandidate(rawToken) + if !isHighEntropySecretToken(token, candidates) { + continue + } + desensitized = strings.ReplaceAll(desensitized, token, "") + } + return desensitized +} + +func highEntropyTokenCandidate(rawToken string) string { + token := normalizeSecretToken(rawToken) + if key, value, ok := strings.Cut(token, "="); ok && isAssignmentKey(key) && value != "" { + return normalizeSecretToken(value) + } + return token +} + +func isAssignmentKey(text string) bool { + for _, r := range text { + switch { + case r >= 'A' && r <= 'Z' || r >= 'a' && r <= 'z' || r >= '0' && r <= '9' || r == '_': + default: + return false + } + } + return text != "" +} + +func isHighEntropySecretToken(token string, candidates []fieldCandidate) bool { + if len(token) < 32 { + return false + } + if isKnownBaseURLValue(token, candidates) || looksLikeURL(token) { + return false + } + + hasLetter := false + hasDigit := false + for _, r := range token { + switch { + case r >= 'A' && r <= 'Z' || r >= 'a' && r <= 'z': + hasLetter = true + case r >= '0' && r <= '9': + hasDigit = true + case r == '+' || r == '/' || r == '=' || r == '_' || r == '-': + default: + return false + } + } + return hasLetter && hasDigit +} + +func isKnownBaseURLValue(token string, candidates []fieldCandidate) bool { + for _, candidate := range candidates { + if candidate.kind == fieldBaseURL && candidate.value == token { + return true + } + } + return false +} + +func looksLikeURL(token string) bool { + // The high-entropy fallback intentionally leaves URL-looking values alone, + // even if they contain token-like path text, so base_url extraction is not + // damaged before downstream provider inference. Bare base64 can contain '/', + // so only clear URLs or path-shaped values are excluded here. + return strings.Contains(token, "://") || strings.HasPrefix(token, "/") +} + +func secretShapePatterns() []string { + return []string{ + `sk-[A-Za-z0-9][A-Za-z0-9._=/+-]{3,}`, + `[A-Za-z0-9_-]{10,}\.[A-Za-z0-9_-]{10,}\.[A-Za-z0-9_-]{10,}`, + `(?i)(?:xox[baprs]?|gh[pousr]|pat|token)[_-][A-Za-z0-9][A-Za-z0-9._=-]{7,}`, + `(?i)[A-Za-z0-9._-]{8,}token[A-Za-z0-9._-]{8,}`, + `AIza[0-9A-Za-z_-]{35}`, + `(?i)[0-9a-f]{40}`, + } +} + +func normalizeSecretToken(secret string) string { + return strings.Trim(strings.TrimSpace(secret), `"'`+"`"+`.,;:()[]{}<>`) +} diff --git a/internal/blobparse/parse_test.go b/internal/blobparse/parse_test.go new file mode 100644 index 0000000..63131b8 --- /dev/null +++ b/internal/blobparse/parse_test.go @@ -0,0 +1,338 @@ +package blobparse + +import ( + "reflect" + "regexp" + "strings" + "testing" +) + +func TestParseExtractsFieldsAndDesensitizes(t *testing.T) { + tests := []struct { + name string + input string + wantBaseURL string + wantAPIKey string + wantModel string + wantSmallFastModel string + wantProvider string + }{ + { + name: "export anthropic env", + input: strings.Join([]string{ + `export ANTHROPIC_BASE_URL=https://api.example.com`, + `export ANTHROPIC_AUTH_TOKEN=sk-ant-secret123`, + `export ANTHROPIC_MODEL=claude-sonnet-4`, + `export ANTHROPIC_SMALL_FAST_MODEL=claude-haiku-4-5`, + }, "\n"), + wantBaseURL: "https://api.example.com", + wantAPIKey: "sk-ant-secret123", + wantModel: "claude-sonnet-4", + wantSmallFastModel: "claude-haiku-4-5", + wantProvider: "anthropic", + }, + { + name: "prose labels", + input: `Base URL: https://proxy.example.test/v1, API Key: sk-prose-secret123, model: claude-opus-4`, + wantBaseURL: "https://proxy.example.test/v1", + wantAPIKey: "sk-prose-secret123", + wantModel: "claude-opus-4", + wantProvider: "", + }, + { + name: "json codex", + input: `{"OPENAI_BASE_URL":"https://compat.example.com/v1","OPENAI_API_KEY":"sk-openai-secret123","CODEX_MODEL":"gpt-4.1","CODEX_MODEL_PROVIDER":"openai"}`, + wantBaseURL: "https://compat.example.com/v1", + wantAPIKey: "sk-openai-secret123", + wantModel: "gpt-4.1", + wantProvider: "openai", + }, + { + name: "yaml anthropic api key fallback", + input: "ANTHROPIC_BASE_URL: https://anthropic.example.com\nANTHROPIC_API_KEY: sk-api-secret123\n", + wantBaseURL: "https://anthropic.example.com", + wantAPIKey: "sk-api-secret123", + wantProvider: "anthropic", + }, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + got := Parse(tt.input) + + if got.Core.BaseURL != tt.wantBaseURL { + t.Errorf("Core.BaseURL = %q, want %q", got.Core.BaseURL, tt.wantBaseURL) + } + if got.Core.APIKey != tt.wantAPIKey { + t.Errorf("Core.APIKey = %q, want %q", got.Core.APIKey, tt.wantAPIKey) + } + if got.Core.Model != tt.wantModel { + t.Errorf("Core.Model = %q, want %q", got.Core.Model, tt.wantModel) + } + if got.Core.SmallFastModel != tt.wantSmallFastModel { + t.Errorf("Core.SmallFastModel = %q, want %q", got.Core.SmallFastModel, tt.wantSmallFastModel) + } + if got.Core.Provider != tt.wantProvider { + t.Errorf("Core.Provider = %q, want %q", got.Core.Provider, tt.wantProvider) + } + if strings.Contains(got.Desensitized, tt.wantAPIKey) { + t.Fatalf("Desensitized leaked api key: %q", got.Desensitized) + } + if len(got.CapturedSecrets) != 1 { + t.Fatalf("CapturedSecrets len = %d, want 1: %#v", len(got.CapturedSecrets), got.CapturedSecrets) + } + placeholder := onlyPlaceholder(t, got.CapturedSecrets) + if got.CapturedSecrets[placeholder] != tt.wantAPIKey { + t.Errorf("CapturedSecrets[%q] = %q, want %q", placeholder, got.CapturedSecrets[placeholder], tt.wantAPIKey) + } + if !strings.Contains(got.Desensitized, placeholder) { + t.Errorf("Desensitized = %q, want placeholder %q", got.Desensitized, placeholder) + } + }) + } +} + +func TestParseCapturesBareSecret(t *testing.T) { + got := Parse("paste this bare key sk-xxxx into the provider form") + + if got.Core.APIKey != "" { + t.Errorf("Core.APIKey = %q, want empty without a label", got.Core.APIKey) + } + if len(got.CapturedSecrets) != 1 { + t.Fatalf("CapturedSecrets len = %d, want 1: %#v", len(got.CapturedSecrets), got.CapturedSecrets) + } + placeholder := onlyPlaceholder(t, got.CapturedSecrets) + if got.CapturedSecrets[placeholder] != "sk-xxxx" { + t.Errorf("CapturedSecrets[%q] = %q, want sk-xxxx", placeholder, got.CapturedSecrets[placeholder]) + } + if strings.Contains(got.Desensitized, "sk-") { + t.Fatalf("Desensitized leaked sk token: %q", got.Desensitized) + } +} + +func TestParseRedactsSecretNamedFieldsWithoutSecretShape(t *testing.T) { + got := Parse("ANTHROPIC_AUTH_TOKEN=plain-token-value\nOPENAI_BASE_URL=https://compat.example.com/v1") + + if got.Core.APIKey != "plain-token-value" { + t.Errorf("Core.APIKey = %q, want plain-token-value", got.Core.APIKey) + } + if strings.Contains(got.Desensitized, "plain-token-value") { + t.Fatalf("Desensitized leaked secret-named field: %q", got.Desensitized) + } + if len(got.CapturedSecrets) != 1 { + t.Fatalf("CapturedSecrets len = %d, want 1: %#v", len(got.CapturedSecrets), got.CapturedSecrets) + } +} + +func TestParseNoRecognizableFieldsPreservesPlainText(t *testing.T) { + input := "hello there\nthis blob has no profile fields" + + got := Parse(input) + + if got.Core.BaseURL != "" || got.Core.APIKey != "" || got.Core.Model != "" || got.Core.SmallFastModel != "" || got.Core.Provider != "" { + t.Fatalf("Core = %#v, want empty", got.Core) + } + if len(got.CapturedSecrets) != 0 { + t.Fatalf("CapturedSecrets = %#v, want empty", got.CapturedSecrets) + } + if got.Desensitized != input { + t.Errorf("Desensitized = %q, want input unchanged", got.Desensitized) + } +} + +func TestParseReusesStablePlaceholderForRepeatedSecret(t *testing.T) { + input := "ANTHROPIC_AUTH_TOKEN=sk-repeat-secret and API Key: sk-repeat-secret" + + got := Parse(input) + + if len(got.CapturedSecrets) != 1 { + t.Fatalf("CapturedSecrets len = %d, want 1: %#v", len(got.CapturedSecrets), got.CapturedSecrets) + } + placeholder := onlyPlaceholder(t, got.CapturedSecrets) + if count := strings.Count(got.Desensitized, placeholder); count != 2 { + t.Errorf("placeholder count = %d, want 2 in %q", count, got.Desensitized) + } +} + +func TestParseStripsResidualSecretShapes(t *testing.T) { + input := "unmapped jwt aaaabbbbcccc.ddddeeeeffff.gggghhhhiiii and token token_secret_value_123456789" + + got := Parse(input) + + assertNoSecretShapes(t, got.Desensitized) +} + +func TestParseDesensitizesExpandedBareSecretShapes(t *testing.T) { + secrets := []string{ + "sk-ant-secret123", + "eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiIxMjM0NTY3ODkwIn0.signedpayload123", + "xoxb-1234567890abcdef", + "ghp_1234567890abcdef", + "pat_1234567890abcdef", + "token_secret_value_123456789", + "AIzaSyA123456789012345678901234567890123", + "0123456789abcdef0123456789abcdef01234567", + "QUJDREVGR0hJSktMTU5PUFFSU1RVVldYWVo1234567890+/=", + } + input := strings.Join([]string{ + `first "` + secrets[0] + `", second (` + secrets[1] + `);`, + secrets[2] + " " + secrets[3] + " " + secrets[4], + "`" + secrets[5] + "` " + secrets[6] + ".", + "hex=[" + secrets[7] + "] b64=" + secrets[8] + ",", + }, "\n") + + got := Parse(input) + + for _, secret := range secrets { + if strings.Contains(got.Desensitized, secret) { + t.Fatalf("Desensitized leaked %q: %q", secret, got.Desensitized) + } + } + assertNoSecretShapes(t, got.Desensitized) +} + +func TestParseHighEntropyFallbackPreservesURLs(t *testing.T) { + baseURL := "https://compat.example.com/v1" + input := "OPENAI_BASE_URL=" + baseURL + " blob QUJDREVGR0hJSktMTU5PUFFSU1RVVldYWVo1234567890+/=" + + got := Parse(input) + + if got.Core.BaseURL != baseURL { + t.Fatalf("Core.BaseURL = %q, want %q", got.Core.BaseURL, baseURL) + } + if !strings.Contains(got.Desensitized, baseURL) { + t.Fatalf("Desensitized removed base URL: %q", got.Desensitized) + } + assertNoSecretShapes(t, got.Desensitized) +} + +func TestParsePropertyNoSecretShapeSurvives(t *testing.T) { + corpus := []string{ + `export ANTHROPIC_BASE_URL=https://api.example.com ANTHROPIC_AUTH_TOKEN=sk-ant-property123`, + `Base URL: https://proxy.example.com API Key: sk-label-property123`, + `bare sk-xxxx should still be removed`, + `{"OPENAI_API_KEY":"sk-json-property123","OPENAI_BASE_URL":"https://compat.example.com/v1"}`, + `plain text with no token`, + `oauth token: token_secret_value_123456789`, + `jwt eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiIxMjM0NTY3ODkwIn0.signedpayload123`, + `google "AIzaSyA123456789012345678901234567890123"`, + `hex (0123456789abcdef0123456789abcdef01234567), next`, + `base64 QUJDREVGR0hJSktMTU5PUFFSU1RVVldYWVo1234567890+/=;`, + `multiple sk-first-secret123 xoxb-1234567890abcdef ghp_1234567890abcdef`, + `TOKEN_SECRET_VALUE_123456789 and pat_1234567890abcdef`, + `wrapped ['AIzaSyA123456789012345678901234567890123'].`, + } + + for _, blob := range corpus { + t.Run(blob, func(t *testing.T) { + got := Parse(blob) + assertNoSecretShapes(t, got.Desensitized) + }) + } +} + +func TestParseInfersProviderFromSelectedAPIKey(t *testing.T) { + input := strings.Join([]string{ + "OPENAI_BASE_URL=https://compat.example.com/v1", + "OPENAI_API_KEY=sk-openai-secret123", + "ANTHROPIC_AUTH_TOKEN=sk-anthropic-secret123", + }, "\n") + + got := Parse(input) + + if got.Core.BaseURL != "https://compat.example.com/v1" { + t.Errorf("Core.BaseURL = %q, want OpenAI-compatible base URL", got.Core.BaseURL) + } + if got.Core.APIKey != "sk-anthropic-secret123" { + t.Errorf("Core.APIKey = %q, want selected Anthropic auth token", got.Core.APIKey) + } + if got.Core.Provider != "anthropic" { + t.Errorf("Core.Provider = %q, want anthropic", got.Core.Provider) + } +} + +func TestParseDoesNotTrimTrailingPunctuationFromAPIKey(t *testing.T) { + input := "ANTHROPIC_AUTH_TOKEN=sk-key-ending. OPENAI_BASE_URL=https://compat.example.com/v1." + + got := Parse(input) + + if got.Core.APIKey != "sk-key-ending." { + t.Errorf("Core.APIKey = %q, want trailing punctuation preserved", got.Core.APIKey) + } + if got.Core.BaseURL != "https://compat.example.com/v1" { + t.Errorf("Core.BaseURL = %q, want trailing punctuation trimmed", got.Core.BaseURL) + } + if strings.Contains(got.Desensitized, "sk-key-ending.") { + t.Fatalf("Desensitized leaked api key with trailing punctuation: %q", got.Desensitized) + } +} + +func TestParseDeterministic(t *testing.T) { + input := `export ANTHROPIC_AUTH_TOKEN=sk-deterministic123 +API Key: sk-second-deterministic` + + first := Parse(input) + second := Parse(input) + + if !reflect.DeepEqual(first, second) { + t.Fatalf("Parse is not deterministic:\nfirst=%#v\nsecond=%#v", first, second) + } +} + +func onlyPlaceholder(t *testing.T, secrets map[string]string) string { + t.Helper() + for placeholder := range secrets { + return placeholder + } + t.Fatal("no placeholder") + return "" +} + +func assertNoSecretShapes(t *testing.T, text string) { + t.Helper() + for _, detector := range testSecretShapeDetectors() { + matches := detector.re.FindAllStringSubmatch(text, -1) + for _, match := range matches { + if len(match) < 3 { + continue + } + secret := match[2] + if detector.mixedAlphaNum && !hasTestLetterAndDigit(secret) { + continue + } + t.Fatalf("desensitized text contains secret shape %q: %q", detector.name, text) + } + } +} + +type testSecretShapeDetector struct { + name string + re *regexp.Regexp + mixedAlphaNum bool +} + +func testSecretShapeDetectors() []testSecretShapeDetector { + return []testSecretShapeDetector{ + {name: "sk prefix", re: regexp.MustCompile(`(^|[^A-Za-z0-9_-])(sk-[A-Za-z0-9][A-Za-z0-9._=/+-]{3,})`)}, + {name: "jwt", re: regexp.MustCompile(`(^|[^A-Za-z0-9_-])([A-Za-z0-9_-]{10,}\.[A-Za-z0-9_-]{10,}\.[A-Za-z0-9_-]{10,})`)}, + {name: "xox gh pat token prefix", re: regexp.MustCompile(`(?i)(^|[^A-Za-z0-9_-])((?:xox[baprs]?|gh[pousr]|pat|token)[_-][A-Za-z0-9][A-Za-z0-9._=-]{7,})`)}, + {name: "embedded token", re: regexp.MustCompile(`(?i)(^|[^A-Za-z0-9_-])([A-Za-z0-9._-]{8,}token[A-Za-z0-9._-]{8,})`)}, + {name: "google api key", re: regexp.MustCompile(`(^|[^A-Za-z0-9_-])(AIza[0-9A-Za-z_-]{35,})`)}, + {name: "40 hex", re: regexp.MustCompile(`(?i)(^|[^A-Za-z0-9_-])([0-9a-f]{40})(?:$|[^0-9a-f])`)}, + {name: "high entropy token", re: regexp.MustCompile(`(^|[^A-Za-z0-9_-])([A-Za-z0-9+/=_-]{32,})`), mixedAlphaNum: true}, + } +} + +func hasTestLetterAndDigit(text string) bool { + hasLetter := false + hasDigit := false + for _, r := range text { + switch { + case r >= 'A' && r <= 'Z' || r >= 'a' && r <= 'z': + hasLetter = true + case r >= '0' && r <= '9': + hasDigit = true + } + } + return hasLetter && hasDigit +}