Repository navigation
Conversation
- Add secret sources that sync key/value pairs from Doppler, Infisical, Vault, 1Password Connect, and generic HTTP or container providers into variables - Scope each source to all environments or selected ones - Keep credentials out of API responses - Add migration 00117 granting secret-sources permissions to full admins and their API keys - Add secret source API routes, `/secrets` page, and CLI `variables sources` commands
- Require secret sources create permission to test an unsaved source - Require update permission only when a saved test changes fetch settings - Update saved sources by ID and return not found when no row matches - Add a switch to clear a saved optional credential in the source form - Reset the refresh button's loading state before showing the result - Print JSON from `secrets sources test` when JSON output is enabled
- Check the error from w.Write in serveSecrets instead of discarding it - Panic on failure so lint no longer flags the ignored error
- Move external secret sources into a Secrets tab on the Variables page - Remove the standalone /secrets route, its sidebar entry, and its auth prefix - Gate the Variables page on variables or secret-sources permissions and show each tab only when its permission is granted - Rename the Variables nav item and page to "Variables & Secrets"
- Wrap the variables page parallel loads in tryCatch - Route load failures through throwPageLoadError so the page error boundary renders them
05d7b71 to
0cfc6d4
Compare
…nal-secrets # Conflicts: # backend/internal/database/database_test.go # frontend/messages/en.json
| func (s *Service) cachedEntries(ctx context.Context, record SecretSource) ([]types.EnvVariable, error) { | ||
| if entries, ok, _ := s.entries.Get(record.ID); ok { | ||
| return withInject(entries, record.InjectIntoContainers), nil |
There was a problem hiding this comment.
Secrets reach wrong agents. An in-flight sync can pass the scope check for agent A before an edit moves the source to agent B and changes its secret path. After the edit refreshes the cache, this lookup can return B's new secrets to the sync for A. The cache-miss path also reloads the source without rechecking its scope. Read the scope and values from the same locked source settings before returning them.
How this was verified: EffectiveLayers checks an earlier row's scope, cachedEntries reads values keyed only by source ID, and SyncEnvironment sends those values to the already-selected agent.
Prompt To Fix With AI
This is a comment left during a code review.
Path: backend/internal/variable/children/source/service.go
Line: 402-404
Comment:
**Secrets reach wrong agents.** An in-flight sync can pass the scope check for agent A before an edit moves the source to agent B and changes its secret path. After the edit refreshes the cache, this lookup can return B's new secrets to the sync for A. The cache-miss path also reloads the source without rechecking its scope. Read the scope and values from the same locked source settings before returning them.
**How this was verified:** `EffectiveLayers` checks an earlier row's scope, `cachedEntries` reads values keyed only by source ID, and `SyncEnvironment` sends those values to the already-selected agent.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.| function credentialIsSet(field: SecretSourceField): boolean { | ||
| return !!sourceToEdit && sourceToEdit.credentialKeys.includes(field.key); | ||
| } |
There was a problem hiding this comment.
Input edits erase saved tokens. Changing the command on a saved
generic-container source makes mergeConfig drop its stored credentials. The sheet still shows the token as saved and omits untouched credentials from buildInput. Because this token is optional, saving succeeds and silently erases it. Require re-entry or an explicit clear before saving an edit that drops a stored credential.
Knowledge Base Used: Web application experience
Prompt To Fix With AI
This is a comment left during a code review.
Path: frontend/src/lib/components/sheets/secret-source-form-sheet.svelte
Line: 111-113
Comment:
**Input edits erase saved tokens.** Changing the command on a saved `generic-container` source makes `mergeConfig` drop its stored credentials. The sheet still shows the token as saved and omits untouched credentials from `buildInput`. Because this token is optional, saving succeeds and silently erases it. Require re-entry or an explicit clear before saving an edit that drops a stored credential.
**Knowledge Base Used:** [Web application experience](https://app.greptile.com/-/switch-tenant?tenant=getarcaneapp&next=%2Fcustom-context%2Fknowledge-base%2Fgetarcaneapp%2Farcane%2F-%2Fdocs%2Fweb-application.md)
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.| defer func() { | ||
| removeCtx, removeCancel := context.WithTimeout(context.WithoutCancel(ctx), timeouts.DefaultDockerAPI) | ||
| defer removeCancel() | ||
| if _, removeErr := dockerClient.ContainerRemove(removeCtx, created.ID, client.ContainerRemoveOptions{Force: true}); removeErr != nil && !cerrdefs.IsNotFound(removeErr) { |
There was a problem hiding this comment.
Fetches leave unused volumes. If the source image declares
VOLUME, Docker creates anonymous volumes for each run. This cleanup only sets Force, so those volumes survive container removal after tests and refreshes. Set RemoveVolumes for these temporary containers.
Prompt To Fix With AI
This is a comment left during a code review.
Path: backend/pkg/secretsource/container.go
Line: 122
Comment:
**Fetches leave unused volumes.** If the source image declares `VOLUME`, Docker creates anonymous volumes for each run. This cleanup only sets `Force`, so those volumes survive container removal after tests and refreshes. Set `RemoveVolumes` for these temporary containers.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.| const result = await tryCatch(secretSourceService.validateDefinition(customYaml)); | ||
| validating = false; | ||
| if (result.error) { | ||
| customDefinition = null; | ||
| definitionError = extractApiErrorMessage(result.error); | ||
| return null; | ||
| } | ||
| customDefinition = result.data; |
There was a problem hiding this comment.
Old checks replace current fields. An older YAML check can finish after a newer one and overwrite
customDefinition. Then resolveDefinition trusts the old result, while buildInput sends the current YAML with fields from the old definition. Users can see outdated fields or have a valid save rejected. Ignore responses that no longer match the current text and check.
Knowledge Base Used: Web application experience
Prompt To Fix With AI
This is a comment left during a code review.
Path: frontend/src/lib/components/sheets/secret-source-form-sheet.svelte
Line: 128-135
Comment:
**Old checks replace current fields.** An older YAML check can finish after a newer one and overwrite `customDefinition`. Then `resolveDefinition` trusts the old result, while `buildInput` sends the current YAML with fields from the old definition. Users can see outdated fields or have a valid save rejected. Ignore responses that no longer match the current text and check.
**Knowledge Base Used:** [Web application experience](https://app.greptile.com/-/switch-tenant?tenant=getarcaneapp&next=%2Fcustom-context%2Fknowledge-base%2Fgetarcaneapp%2Farcane%2F-%2Fdocs%2Fweb-application.md)
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.| }, | ||
| } | ||
| bindSourceFlags(command, &opts) | ||
| _ = command.MarkFlagRequired("name") |
There was a problem hiding this comment.
Flag errors go unexplained. This new call discards the error from
MarkFlagRequired without explanation. The Go guide requires handling every error or adding a comment explaining why it can be ignored. Handle it, or document that bindSourceFlags already registered the flag.
Rule Used: --- name: go-struct-literals description: Write Go 1.27+ in a declarative, composite-literal style. Every value (structs, slices, maps, configs, errors, servers, test tables, constructor results) is built with one keyed composite literal instead of f... (source)
Prompt To Fix With AI
This is a comment left during a code review.
Path: cli/pkg/variables/sources.go
Line: 283
Comment:
**Flag errors go unexplained.** This new call discards the error from `MarkFlagRequired` without explanation. The Go guide requires handling every error or adding a comment explaining why it can be ignored. Handle it, or document that `bindSourceFlags` already registered the flag.
**Rule Used:** --- name: go-struct-literals description: Write Go 1.27+ in a declarative, composite-literal style. Every value (structs, slices, maps, configs, errors, servers, test tables, constructor results) is built with one keyed composite literal instead of f... ([source](https://app.greptile.com/-/switch-tenant?tenant=getarcaneapp&next=%2Fcustom-context%3Fmemory%3D214b40a8-9695-4738-986d-5949b5d65ff1))
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
…nal-secrets # Conflicts: # frontend/src/lib/components/sheets/webhook-form-sheet.svelte # frontend/src/routes/(app)/projects/variables/+page.svelte
| for number, line := range strings.Split(string(content), "\n") { | ||
| line = strings.TrimSpace(line) | ||
| if line == "" || strings.HasPrefix(line, "#") { | ||
| continue | ||
| } | ||
| key, value, ok := strings.Cut(line, "=") |
There was a problem hiding this comment.
File credentials lose spaces.
readCredentials trims each whole line before splitting at =, so a password ending in a space loses that character. Custom definitions can use credentials as passwords, so the source gets saved or tested with the wrong password. Trim only the key and a separate copy used to detect blank or comment lines; preserve the value.
Prompt To Fix With AI
This is a comment left during a code review.
Path: cli/pkg/variables/sources.go
Line: 123-128
Comment:
**File credentials lose spaces.** `readCredentials` trims each whole line before splitting at `=`, so a password ending in a space loses that character. Custom definitions can use credentials as passwords, so the source gets saved or tested with the wrong password. Trim only the key and a separate copy used to detect blank or comment lines; preserve the value.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.| "strings" | ||
| "time" | ||
|
|
||
| cerrdefs "github.com/containerd/errdefs" |
There was a problem hiding this comment.
Imports break the Go guide.
container.go renames github.com/containerd/errdefs to cerrdefs, but no other import conflicts with errdefs. The repository guide requires default import names and allows aliases only to fix a real conflict. Remove this alias and use errdefs.IsNotFound.
Context Used: AGENTS.md (source)
Prompt To Fix With AI
This is a comment left during a code review.
Path: backend/pkg/secretsource/container.go
Line: 17
Comment:
**Imports break the Go guide.** `container.go` renames `github.com/containerd/errdefs` to `cerrdefs`, but no other import conflicts with `errdefs`. The repository guide requires default import names and allows aliases only to fix a real conflict. Remove this alias and use `errdefs.IsNotFound`.
**Context Used:** AGENTS.md ([source](https://github.com/getarcaneapp/arcane/blob/main/AGENTS.md))
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
- Rename secret-sources:* permissions to variables:sources:* in the authz catalog, access policy, routes, webhook refresh mapping, and frontend permission checks - Update migration 00118 and its tests to the new permission strings (feature is unreleased)
- Remove the separate "External Secrets" permission resource - List its actions under Variables as sources-list through sources-refresh - Relabel them "... external secrets" and update the Variables description
Checklist
mainbranchm.*())What This PR Implements
Arcane can now pull variables from an external secrets manager, store them as variables, and sync them to environments and agents. Each source is described by a YAML definition, so a new provider needs a definition file and no new Go code.
Shipped presets: Infisical, Doppler, Vault KV v2 (AppRole and token auth), 1Password Connect, generic HTTP JSON, and generic container. Custom definitions can be pasted in and validated before saving.
Addresses discussion #4036 (external secrets managers). This is a discussion, not an issue, so there is no issue number to close.
Changes Made
backend/internal/variable/children/sourceadds create, read, update, delete, refresh, test, definition validation, and preset listing under/api/variables/sources.backend/pkg/secretsourceparses definitions, fetches over HTTP or from a container, extracts keys, renders templates, and loads presets frombackend/resources/secretsources/*.yaml.environment.ReplaceScopeRows.secretsource.secret-sources:*permissions (list, read, create, update, delete, refresh) in the authz catalog.secret_sourcesandsecret_source_environments. It also grants the new permissions to roles and API keys that already hold every prior permission (full admins). Partial roles and keys get nothing. The down migration removes the tables, the grants, and the webhooks targeting sources.backend/pkg/httpx/safe_remote.gois reworked for remote source fetches.projects.EnvKeyPatternreplaces a duplicate regex in the lifecycle service.arcane variables sourcescommands incli/pkg/variables/sources.go.Testing Done
TestMigration117_AddsSecretSourcesAndGrantsFullAdminscovers the up migration, full-admin grants, partial role and key exclusion, name uniqueness, and the down migration.TestMigration102_CleansUpRBACPermissionsnow migrates to 117.backend/internal/variable/children/source/service_test.go.backend/internal/variable/service_test.gois updated for the new variable flow.Not verified yet:
AI Tool Used (if applicable)
Additional Context
secret-sources:*permissions automatically.secret-sourcesnaming in the code.Disclaimer Greptiles Reviews use AI, make sure to check over its work.
To better help train Greptile on our codebase, if the comment is useful and valid Like the comment, if its not helpful or invalid Dislike
To have Greptile Re-Review the changes, mention
greptileai.Not ready to merge: concurrent syncs can still send secrets to the wrong agent, and editing container inputs can erase saved tokens.
Fix with agent prompt
Summary
Adds external secret sources that feed variables across selected environments, with provider presets, custom definitions, webhooks, CLI commands, and a shared Variables & Secrets page.
Verification: compared all 237 catalog permissions with the migration lists and checked permission paths. Browser, live-provider, and related-repository checks were not completed; access to the related repositories was denied.
Reviews (8) · Last reviewed commit: "refactor: move secret source permissions..." · Reviewed by Greptile