Skip to content

feat: manual project backups (files and volumes in one snapshot) - #4894

Open
neurekadev wants to merge 40 commits into
getarcaneapp:mainfrom
neurekadev:feat/4880-project-backups
Open

neurekadev wants to merge 40 commits into
getarcaneapp:mainfrom
neurekadev:feat/4880-project-backups

Conversation

@neurekadev

@neurekadev neurekadev commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

Important

PR 4 of 4. Merge in order: #4876 (merged) → #4879 → #4888 → this PR. It is stacked on #4888, so the diff looks large: it includes #4879's and #4888's changes until those merge. Only this PR's changes.

Checklist

  • This PR is not opened from my fork’s main branch
  • All new user-facing strings are translated via Paraglide (m.*())

What This PR Implements

Part of #4880 (phases 1 and 2 of the plan there). Restore, schedules and the system backup split come in later PRs, so this doesn't close the issue.

A project can now be backed up as one Rustic snapshot holding its files and the data of its volumes, so both come from the same point in time. Backups can be listed and deleted from a new Backups tab on the project page. Volume and system backups behave as before.

Changes Made

Snapshot (internal/project/children/backup, new child domain):

  • One Rustic run per backup through the existing backup.Engine. The project directory is mounted at project/, each selected volume at volumes/<docker volume name>/, and the manifest at meta/arcane-project.json, all read-only.
  • The manifest is feat: export and import projects with their volumes #4888's types.ProjectManifest with kind: "backup". Volume selection is feat: export and import projects with their volumes #4888's ClassifyProjectVolumes: named and local external volumes that exist are included by default; volumes with driver options are listed but not backed up.
  • The manifest is written into a short-lived volume by a container that mounts only that volume, so it avoids the copy problem from System backup fails again when staging Rustic CA certificates (regression of #4616) #4870.
  • Snapshots go into the existing volumes repository with its password and re-key handling, so data already backed up per volume deduplicates. Tags: arcane-kind:project, arcane-project:<id>, arcane-run:<id>.
  • Destinations: local, S3 only, or local then replicated to S3.

Consistency:

  • stopContainers is on by default. The backup takes the project lock (as a consistent export does), stops only this project's running and restarting containers, and starts them again through Compose in dependency order, on every path including errors.
  • The stopped containers are saved in the workflow checkpoint, so a resumed run restarts them before doing anything else.
  • Arcane's own containers and other containers are never stopped. A volume one of them mounts is marked in use, and the manifest is then not consistent.

Admission: scope project-backup per project, plus the volume-backup lease and volume lock for each captured volume, so a project backup can't overlap a per-volume backup or restore of the same volume. Manual runs fail fast with a conflict.

Shared plumbing moved into internal/backup (no behaviour change): the volumes repository builders and password, container stop/start for volume backups, and source-mount resolution used by system snapshots. Volume discovery now skips arcane-kind:project snapshots, so they never show up as volume backups.

API (/agents/{id}/projects/{projectId}/backups): POST ({volumes?, stopContainers?, destination?, s3DestinationId?}), GET list (paginated, newest first), GET /{backupId}, DELETE /{backupId} (forgets the snapshots and deletes the row). The routes are on the agent allowlist, so remote environments work.

Permissions and data: new projects:backup. Migration 00118 creates project_backups and grants projects:backup to roles and API keys that already have volumes:backup and projects:read, with a real down migration. Deleting an S3 destination counts project backups as in use.

UI: a Backups tab (the existing backup tab is the Git push-back feature) listing date, status, volumes, destination and size, with a mobile card layout. Back up now opens a dialog with the same volume rules as the export dialog, the stop-containers toggle and the destination select. Delete asks for confirmation.

Testing Done

  • just format all, just lint all (0 issues; svelte-check 0 errors), go build -tags exclude_frontend ./....
  • Tests pass for every touched backend package.
  • New tests:
    • migration 118 up/down and its permission grants
    • snapshot input: mounts, tags and as-path for named, external and missing volumes
    • manifest kind and the consistent rule
    • volume discovery skipping project snapshots
    • admission conflict failing fast
    • a resumed run restarting stopped containers before failing
    • stopping and restarting only this project, through Compose, leaving paused, Arcane and other containers alone
    • the moved internal/backup helpers (their tests moved with them)
  • Ran end to end against an image built from this branch with a seeded test project (see Evidence): the backup stopped and restarted only the project's two containers under the same IDs, left a standalone container that mounts the shared volume running, captured the files and all three volumes, and wrote consistent: false because of that container.

AI Tool Used (if applicable)

Claude Code (Claude Opus 5.5) planned and implemented this, ran the local review and the evidence, and drafted this description, at my direction.

Additional Context

🤖 Generated with Claude Code

📸 Evidence

Surfaces: 4 Captured: 4 Open: 0 Recording: 2

Before main @ 8612ea241 → After feat/4880-project-backups @ fb62d63e2 · Arcane with a seeded test project evidence-shop (nginx + busybox; named volumes site and cache, and a local external volume evidence-shared) on a sandbox Docker engine, plus evidence-other, a standalone container that also mounts evidence-shared; test account only, no private data. After captured from fb62d63; Before is main, because #4888 doesn't change the project tabs

Surface Viewport Capture Look for
🖥️ Project tabs Desktop 1280×800 Before / After New Backups tab next to Git Backup
🎬 Backups tab and Back up now dialog Desktop 1280×800 Recording 0:07 Backups listed with date, status, volumes, destination and size; the dialog offers the local external volume with the in-use warning, the stop-containers toggle and the destination
🎬 Backups tab and dialog on a phone Phone 390×844 Recording 0:09 Backups as cards, dialog as a bottom sheet, no horizontal overflow
🔌 Backup API and snapshot — Response excerpt Only this project's containers restarted (same IDs); files and all three volumes in one snapshot; consistent false because another container uses evidence-shared

🖥️ Project tabs — desktop

Before · 8612ea241 After · fb62d63e2
Before: Project tabs After: Project tabs
🎬 Backups tab and Back up now dialog — recording, 0:07 After: Backups tab and Back up now dialog

▶️ Full-quality MP4 (132 KB)

🎬 Backups tab and dialog on a phone — recording, 0:09 After: Backups tab and dialog on a phone

▶️ Full-quality MP4 (133 KB)

🔌 API response — POST /api/agents/0/projects/{id}/backups {}, trimmed to changed fields
+ status succeeded, destination local, stoppedContainers true, volumes cache, shared, site
+ evidence-shop-web-1 / worker-1: same IDs, Up 1 second; evidence-other: Up 33 minutes (not stopped)
+ snapshot 40e0aa31 label=evidence-shop tags=arcane-kind:project,arcane-project:<id>,arcane-run:<backup id>
+ contents: meta/arcane-project.json, project/compose.yaml, volumes/evidence-shared/asset.txt, volumes/evidence-shop_cache, volumes/evidence-shop_site/{50x,index}.html
+ manifest: kind backup, formatVersion 1, shared inUse true, consistent false

📸 Captured by Claude Opus 5.5 (High) · T3 Code (Claude Code) · Playwright · Chromium (mcr.microsoft.com/playwright, --docker); Arcane images built with docker/Dockerfile; rustic v0.11.2 for the snapshot listing · 2026-11-19 02:25 UTC

Disclaimer Greptiles Reviews use AI, make sure to check over its work.

To better help train Greptile on our codebase, if the comment is useful and valid Like the comment, if its not helpful or invalid Dislike

To have Greptile Re-Review the changes, mention greptileai.

RetriggerView in GreptileConfidence Score: 1/5 Tier: plus

Not ready to merge: a crash during backup can still leave project containers stopped.

Fix All in CodexFindings

  1. P1 Crashes can leave projects stopped. ▶
  2. P1 Queued backups use old volumes. ▶
  3. P1 Active S3 destinations remain editable. ▶
  4. P1 Imports change Compose overrides. ▶
  5. P2 Bundle code ignores feature layout ▶
  6. P2 Mixed scopes miss backup grants. ▶
  7. P2 Plan fields change after creation. ▶
  8. P2 Unused activity dependency. ▶
Fix with agent prompt
### Issue 1
backend/internal/project/children/backup/service.go:551-553
The recovery list is saved only after `StopContainers` returns. If Arcane is killed while that call is stopping a multi-container project, Docker has already stopped some containers but the saved `Stopped` list is still empty. The resumed run cannot tell those containers need restarting. Save the recovery set before issuing the stops.

### Issue 2
backend/internal/project/children/backup/service.go:508-511
`StartBackup` freezes `p.Volumes` before taking the project lock, but `capture` reloads the project's current files. A compose edit and deploy can finish while the backup is queued or waiting for that lock. If they switch the project to another volume, the backup can succeed with the new compose file and the old volume's data. Check the plan against the locked project before capture, then reject or rebuild it with matching volume locks and leases.

### Issue 3
backend/internal/s3/service.go:undefined-262
On the first manual backup to a destination, `s3_destination_id` is set but `remote_snapshot_id` stays empty until the S3 write finishes. This filter lets the destination change or be deleted during that write. The snapshot can finish in the old bucket while its row points at the changed or deleted destination, leaving the backup unreachable. Count running project backups as references too.

### Issue 4
backend/pkg/projects/env.go:836-839
This equality check loses whether `COMPOSE_FILE` explicitly selected one file. For a source with `.env.global` setting `COMPOSE_FILE=compose.yaml` and an unused `compose.override.yaml`, the bundle records only `compose.yaml`. On a destination without that global setting, this early return leaves `COMPOSE_FILE` unset, so Compose now merges the override. The import can succeed with different images, commands, or settings. Preserve the exact loaded file set and whether automatic overrides apply.

### Issue 5
backend/internal/project/service.go:1832
**Bundle code ignores feature layout.** `projectBundles` adds export, import, volume restore, and rollback logic to the parent `service.go`. `AGENTS.md` requires substantial features under `children/<feature>/` and one job per file. Move this bundle code into a project child and keep the parent focused on connecting it to project operations.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

### Issue 6
backend/internal/database/migrations/00118_project_backups.go:46-49
The new grant requires matching `environment_id` values, but permission checks combine global and per-environment grants. A scoped key with global `projects:read` and `volumes:backup` for one environment has both prerequisites there, yet gets no `projects:backup` row. Handle these mixed scopes in both SQLite and Postgres so the migration grants the permission over their shared environments.

### Issue 7
backend/internal/project/children/backup/service.go:206-207
`resolvePlan` builds `p`, then fills `S3DestinationID`, `Volumes`, and the inspected volume fields afterward. The Go rule requires one keyed literal per value, with known fields set inside it. Resolve and validate these inputs first, then build the inspected entries and final `plan` with literals.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

### Issue 8
backend/internal/project/service.go:undefined-208
The new `projectBundles.activity` field is never read, but `New` and the tests still wire it in. Remove the field, its import, and its wiring so bundle code does not advertise a dependency it never uses.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Summary

Adds manual project backups that keep project files and selected volume data in one snapshot, with local and S3 storage and a Backups tab.

  • Project backups keep project files and selected volume data in one snapshot.
  • Project exports and imports can carry project files and volume archives together.
  • Project, volume, and system backups share repository and container helpers.

Review was static; tests were not rerun.

Reviews (2) · Last reviewed commit: "fix: treat restarting containers as runn..." · Reviewed by Greptile

neurekadev and others added 30 commits November 18, 2026 11:07
…ping

Scheduled system backups and system-managed volume backups share one
admission and both default to 03:00. The one that started second was
skipped, and the Activity Center reported the skip as a success.

Scheduled runs now wait for the admission, up to six hours, and fail
visibly if it never frees. Manual runs still return a conflict right away.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ention

Scheduled runs now take the admission before reading their policy, so a
policy edited or disabled during the wait is respected. Retention and runs
resumed after a restart wait for the admission too. Before this, retention
failed and marked a successful backup as failed whenever the other
scheduled job grabbed the admission first, and a resumed backup could lose
the admission to a redelivered waiting job.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…starts

A manual system-managed volume backup started while another backup runs
returns not-found or validation errors again instead of the conflict.
Scheduled runs already hold the admission before calling prepare.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Manual system and volume backup workflows run one task at a time, and a
new manual start parks the admission for its queued task. A resumed manual
run that waited for the admission would block on that task until the
six-hour timeout, so manual resumes fail fast again. Scheduled policy
workflows never park the admission, so their resumes still wait.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A scheduled backup of a volume that another backup holds was skipped and
reported as succeeded, and a scheduled system-managed run only counted
that volume as skipped. Scheduled backups now wait for the volume's
admission. Manual backups still fail fast, because a manual start parks
the admission for its queued task.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…olume

Scheduled per-volume and system-managed backups now take the volume
admission before resolving their plan, so a policy edited, disabled, or
deleted during the wait is respected. A per-volume policy disabled or
deleted meanwhile ends its run as canceled, and a volume-level policy
enabled meanwhile takes the volume out of the system-managed run. Manual
starts keep their validation, authorization, then conflict order.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Project export can now include the project's named volumes in one
portable bundle: a gzip tar with an arcane-project.json manifest, the
existing project archive, and one existing volume archive per Compose
key. The project can be stopped during export for a consistent copy.
Import detects a bundle, refuses before changing anything if a target
volume already exists, recreates the project and its volumes by Compose
key, rolls back on a failed volume restore, and can start the project.
Plain exports and plain project archives behave as before.

The volume classification shared with project transfer moves to
pkg/projects, so transfer, export, and the planned project backups use
one rule. The UI gets Export on the project page and Import on the
projects list; the CLI gets `projects export` and `projects import`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The export preview marks named volumes that don't exist yet as missing,
so default exports of undeployed projects no longer fail. An explicitly
requested missing volume is rejected before the project is stopped.

Exports are written straight into their upload session instead of a
temp file that was then copied, so they need one copy of disk space.
Browsers download them through a new streaming upload-session download
route instead of holding the whole bundle in memory; the session is
removed once it is fully sent. Proxied upload chunk reads and downloads
now check the upload kind's permission, so remote exports work for
non-admin users. The CLI refuses a plain archive from a server that
ignores the volume request.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An empty named volume exports as an archive with only its root
directory. The volume restore rejects empty uploads, so importing such a
bundle failed and rolled the whole project back. Import now recognizes
an empty volume archive and creates the Compose-labelled volume
directly, tracked for rollback like restored volumes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
getarcaneapp#4876 merged as 40e65a7. Resolved the conflict by keeping upstream's
removal of the admissionWait comment.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Brings in upstream main through getarcaneapp#4879. Resolved the en.json conflict by
keeping upstream's new keys, then this branch's keys appended after them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Import was a menu action, which folded Update Projects into an Actions
menu with it and broke the E2E tests that click Update Projects. Import
is now a secondary button, so the projects header keeps its existing
buttons.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A project backup is one Rustic snapshot of the project directory, the
data of its selected volumes, and an arcane-project.json manifest, so
files and volumes come from the same point in time. Containers are
stopped for consistency by default and always started again; a resumed
run restarts them first. Backups go to the shared volumes repository,
where they deduplicate against volume backups, locally, to S3, or both.

The project page gets a Backups tab to create, list, and delete them,
behind a new projects:backup permission (granted by migration 118 where
volumes:backup and projects:read are already held).

The volumes repository, password resolution, container stop/start for
backups, and source mount resolution move into internal/backup so volume,
system, and project backups share them, without behaviour changes.
Volume discovery skips project snapshots.

Refs getarcaneapp#4880

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Back up now button sat in the table toolbar, which the backups table
hides, so backups could not be started from the UI; it now sits next to
the panel's description. S3 destinations referenced by project backups
can no longer be deleted or moved. A delete that only removes the local
snapshot updates the destination shown. Create and delete use the shared
API result handling, and a failure to load S3 destinations is shown.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…d import

Import checks the volume names the imported project actually uses before
restoring anything, and only rolls back volumes it created, so a volume
that existed before can no longer be deleted. Empty volumes keep the
archive root's owner and mode.

Export with stop only stops the project's running containers, holds the
project lock while volumes are copied, and starts exactly those
containers again instead of redeploying the whole project.

The CLI downloads into a temporary file and only replaces the output once
the download completes, so a failed re-export keeps the previous bundle;
pipes such as /dev/stdout are still written directly.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
External volumes on the local driver without driver options are often
just a way to pick the Docker volume name, so they are now included in
exports by default like named volumes. Import recreates them under their
exact name without Compose labels, or keeps an existing one as Compose
would, never restoring into or deleting it. Volumes with driver options
stay excluded.

An export is no longer marked consistent when another project's running
containers use one of its volumes, and the dialog says so. Rollback only
removes volumes this import reports it created, and two volumes that
resolve to the same Docker name are rejected before anything changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Brings in the export/import review fixes and local external volume
support. Project backups now use the same rule: existing named volumes
and local external volumes are captured, and volumes with driver options
are left out. Resolved the en.json conflict by keeping both sets of keys.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An import whose external volume already exists is now refused before
anything changes, like named volumes, instead of reusing the existing
volume. Imports are all or nothing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A consistent export now also stops containers Docker is restarting, so
none can write while volumes are copied. Afterwards the stopped services
are started through Compose, which honours depends_on and health
conditions; containers Compose can't start on their own are started
directly, and everything is started directly if Compose fails.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Compose keys that resolve to the same Docker volume now share one archive
in the bundle, and import restores that volume once, so bundles where two
external keys use the same name import again. External volumes keep
their own Docker labels: export records them and import recreates the
volume with them, still without Compose labels.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Brings in getarcaneapp#4887. Resolved the restore conflict by keeping its archive
root owner and mode handling together with this branch's created
reporting.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Brings in the export restart fixes, shared volume archives, external
volume labels, and upstream main through getarcaneapp#4887.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ports clean

An import now holds the project lock from before the project exists until
its volumes are restored, it is started, or it is rolled back, so nothing
can deploy it mid-restore. A bundle's recorded compose files are applied
to the imported project's .env when the target would pick others, and the
import is refused before anything is created if they can't be. The CLI
prints nothing after an export written to stdout or another pipe.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Exporting the Compose project Arcane runs in with stop enabled would stop
Arcane itself and leave it down. Arcane's containers are now skipped and
count as in use, so such an export is not marked consistent.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Project backups now stop only the project's running and restarting
containers, leave paused ones and other projects alone, and start them
again through Compose so depends_on and health conditions hold. A backup
is not marked consistent when another project's running container uses
one of its volumes. The backup dialog offers the volumes the backend
includes, local external volumes too, and warns about shared ones.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ning

The in-use check counts every running container that the export does
not stop, including standalone containers, not only other projects.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Resolves the in-use wording conflicts and applies the same wording to the
backup dialog, the stopContainers doc, and the backup comments: anything
outside the project keeps running, not only other projects.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A deploy, down, or project edit during a backup could fail against the
stopped containers, change what the snapshot captures, or leave the
restart to start containers without their Compose order. The backup now
takes the project lock before stopping, as a consistent export does, and
a failed restart is no longer retried a second time by the deferred one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@neurekadev

Copy link
Copy Markdown
Contributor Author

🔍 Code Review

PR #4894 · c3c458c88..c9e15aacd · 39 files · ✅ No blocking issues remain

Critical: 0 High: 1 Medium: 3 Low: 11 Informational: 0

# Severity Finding Status
1 High Back up now button never rendered
frontend/src/routes/(app)/projects/components/project-backups-panel.svelte:321 · correctness
✅ Fixed 7816ed757
2 Medium S3 destinations in use by project backups could be deleted or moved
backend/internal/s3/service.go:262 · correctness
✅ Fixed 7816ed757
3 Medium Local external volumes could not be selected in the backup dialog
frontend/src/routes/(app)/projects/components/project-backups-panel.svelte:70 · correctness
✅ Fixed d86891b65
4 Medium Restart ignored Compose dependency order
backend/internal/project/service.go:2164 · reliability
✅ Fixed d86891b65
5 Low No project lock while the project is stopped
backend/internal/project/children/backup/service.go:491 · reliability
✅ Fixed c9e15aacd
6 Low A failed restart ran a second time
backend/internal/project/children/backup/service.go:564 · reliability
✅ Fixed c9e15aacd
7 Low Manifest marked consistent while another container used a captured volume
backend/internal/project/children/backup/service.go:638 · correctness
✅ Fixed d86891b65
8 Low Other projects' containers were stopped too
backend/internal/project/service.go:2037 · correctness
✅ Fixed d86891b65
9 Low Paused containers came back running (pre-existing)
backend/internal/project/service.go:2037 · correctness
✅ Fixed d86891b65
10 Low Delete left the destination label stale after a partial failure
backend/internal/project/children/backup/service.go:774 · correctness
✅ Fixed 7816ed757
11 Low Panel error handling diverged from the shared helper
frontend/src/routes/(app)/projects/components/project-backups-panel.svelte:136 · maintainability
✅ Fixed 7816ed757
12 Low Containers stopped before a crash mid-stop are not restarted on resume (pre-existing)
backend/internal/project/children/backup/service.go:364 · reliability
📝 Reported
13 Low Resume restarts without the project lock
backend/internal/project/children/backup/service.go:366 · reliability
📝 Reported
14 Low Deleting a global variable mid-backup can still break the restart's Compose load
backend/internal/variable/service.go:534 · reliability
📝 Reported
15 Low Backup container stops and starts aren't attributed to the user
backend/internal/project/service.go:2037 · maintainability
📝 Reported
1 · Back up now button never rendered
  • Scenario: Opening the Backups tab shows the list but no way to start a backup.
  • Evidence: The button was passed as customToolbarActions, which ArcaneTable only renders inside its search toolbar, and the panel set withoutSearch.
  • Fix: The button sits in the panel's own header row, gated only by projects:backup. (7816ed757)
  • Fix check: Pass: renders in empty, loading and error states; fits at 320px.
2 · S3 destinations in use by project backups could be deleted or moved
  • Scenario: Deleting or re-pointing an S3 destination that holds project backups succeeds; the rows can then never be deleted, or their remote data is orphaned.
  • Evidence: DestinationInUse counted system and volume backups and policies but not project_backups.
  • Fix: project_backups rows with a remote snapshot count as in use, on the manager and through the agent usage endpoint. (7816ed757)
  • Fix check: Pass: local-only rows don't count; tested.
3 · Local external volumes could not be selected in the backup dialog
  • Scenario: A project with a local external volume: the backend includes it by default, but the dialog disables it, so a UI backup never captures it.
  • Evidence: The panel kept the old named-only rule after the backend adopted feat: export and import projects with their volumes #4888's rule; the export dialog uses included.
  • Fix: The panel uses included, as the export dialog does; texts and API docs updated. (d86891b65)
  • Fix check: Pass.
4 · Restart ignored Compose dependency order
  • Scenario: An app with no restart policy is started before its database, exits, and stays down after the backup.
  • Evidence: Containers were restarted with plain ContainerStart in ContainerList order, newest first.
  • Fix: Restart goes through the project's Compose start in depends_on and health order, the same code export uses. (d86891b65)
  • Fix check: Pass in the normal path; the Compose-load fallback is covered by the project lock below.
Low and informational (11)
  • 5 · No project lock while the project is stopped (backend/internal/project/children/backup/service.go:491)
    • Scenario: A deploy, down or compose edit during a backup fails against stopped containers, changes what is captured while the manifest says consistent, or breaks the Compose load at restart.
    • Evidence: Export holds the project lock during a consistent copy; the backup did not.
    • Fix: The backup takes the project lock before stopping and holds it through the restart, in the same lock order as export. (c9e15aacd)
    • Fix check: Pass: defer order, no self-deadlock, resume re-entry and lock order checked.
  • 6 · A failed restart ran a second time (backend/internal/project/children/backup/service.go:564)
    • Scenario: The explicit restart fails, then the deferred restart runs it again, doubling health waits and errors.
    • Evidence: stopped was only cleared after a successful restart, so the defer saw it again.
    • Fix: stopped is cleared after the explicit attempt either way; the checkpoint keeps the list only on failure. (c9e15aacd)
    • Fix check: Pass.
  • 7 · Manifest marked consistent while another container used a captured volume (backend/internal/project/children/backup/service.go:638)
    • Scenario: A captured volume mounted by a running container outside the project is labelled consistent.
    • Evidence: Consistent only looked at stopContainers and the running count; export also checks in-use volumes.
    • Fix: Included volumes get inUse during the backup and Consistent requires none in use. (d86891b65)
    • Fix check: Pass: tested.
  • 8 · Other projects' containers were stopped too (backend/internal/project/service.go:2037)
    • Scenario: Backing up a project that shares a volume stops another project's containers, contrary to the dialog text.
    • Evidence: The stop scope was every container mounting a captured volume.
    • Fix: Only this project's containers are stopped; shared volumes are reported in use. (d86891b65)
    • Fix check: Pass.
  • 9 · Paused containers came back running (backend/internal/project/service.go:2037)
    • Scenario: A paused service is stopped for the backup and then started, so it ends up running.
    • Evidence: The stop list included paused containers and the restart started everything stopped.
    • Fix: Only running and restarting containers are stopped. (d86891b65)
    • Fix check: Pass.
  • 10 · Delete left the destination label stale after a partial failure (backend/internal/project/children/backup/service.go:774)
    • Scenario: The local forget succeeds and the S3 forget fails; the row still shows local+S3.
    • Evidence: Destination was not recomputed before saving, unlike volume backups.
    • Fix: Destination is recomputed from the remaining snapshot IDs. (7816ed757)
    • Fix check: Pass.
  • 11 · Panel error handling diverged from the shared helper (frontend/src/routes/(app)/projects/components/project-backups-panel.svelte:136)
    • Scenario: S3 destinations fail to load and the destination list is silently empty.
    • Evidence: Create and delete used hand-written tryCatch toasts; AGENTS.md asks for handleApiResultWithCallbacks.
    • Fix: Create and delete use handleApiResultWithCallbacks; the S3 load error is shown as in the volume backup table. (7816ed757)
    • Fix check: Pass.
  • 12 · Containers stopped before a crash mid-stop are not restarted on resume (backend/internal/project/children/backup/service.go:364)
    • Scenario: Arcane dies while stopping a project's containers; the ones already stopped were not yet checkpointed and stay stopped after resume.
    • Evidence: The stopped list is recorded once, after all stops return. Volume backups have the same window and don't restart anything on resume.
    • 📝 Reported: Needs a hard crash inside the stop loop; left for a follow-up.
  • 13 · Resume restarts without the project lock (backend/internal/project/children/backup/service.go:366)
    • Scenario: Right after Arcane restarts, a deploy of the same project overlaps the resumed restart.
    • Evidence: resume calls StartContainers before execute, outside capture's lock.
    • 📝 Reported: Narrow window after a crash; left as is.
  • 14 · Deleting a global variable mid-backup can still break the restart's Compose load (backend/internal/variable/service.go:534)
    • Scenario: The compose file uses ${X:?} and X is deleted during the backup; the restart falls back to direct starts.
    • Evidence: WriteLocalEnvFile doesn't take the project lock.
    • 📝 Reported: Rare; outside this domain.
  • 15 · Backup container stops and starts aren't attributed to the user (backend/internal/project/service.go:2037)
    • Scenario: The event log shows Docker's stop and start events with no user.
    • Evidence: The shared stop/start path calls Docker directly, as export already does.
    • 📝 Reported: Same as export; mentioned in the PR description.

🤖 Claude Opus 5.5 (High) · T3 Code (Claude Code) · Round 2 of 3

neurekadev and others added 5 commits November 18, 2026 18:15
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Brings in main's frontend query cleanup through getarcaneapp#4888. The Backups tab
joins the rebuilt tab list, and the backups list call is typed for the
new API client.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The export dialog fetched its options from an $effect that wrote state.
It now uses a project query enabled while the dialog is open, refetched
on every open so in-use and running state is current.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
neurekadev added a commit to neurekadev/arcane that referenced this pull request Nov 19, 2026
neurekadev and others added 2 commits November 18, 2026 18:27
Brings in getarcaneapp#4893. Transfer keeps the shared volume classification and
skips named volumes missing on the source, as getarcaneapp#4893 does; the restore
keeps getarcaneapp#4893's empty-archive rule and the created-volume result.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Comment on lines +544 to +546
stopped, stopErr = s.deps.StopContainers(ctx, project.ComposeName)
p.Stopped = checkpointContainers(stopped)
if recordErr := errors.Join(stopErr, s.record(ctx, p)); recordErr != nil {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Crashes can leave projects stopped. The recovery list is saved only after StopContainers returns. If Arcane is killed while that call is stopping a multi-container project, Docker has already stopped some containers but the saved Stopped list is still empty. The resumed run cannot tell those containers need restarting. Save the recovery set before issuing the stops.

Prompt To Fix With AI
This is a comment left during a code review.
Path: backend/internal/project/children/backup/service.go
Line: 544-546

Comment:
**Crashes can leave projects stopped.** The recovery list is saved only after `StopContainers` returns. If Arcane is killed while that call is stopping a multi-container project, Docker has already stopped some containers but the saved `Stopped` list is still empty. The resumed run cannot tell those containers need restarting. Save the recovery set before issuing the stops.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Fix in Codex Fix in Claude Code Fix in Cursor

Comment on lines +506 to +509
project, err := s.deps.Project(ctx, entry.ProjectID)
if err != nil {
return err
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Queued backups use old volumes. StartBackup freezes p.Volumes before taking the project lock, but capture reloads the project's current files. A compose edit and deploy can finish while the backup is queued or waiting for that lock. If they switch the project to another volume, the backup can succeed with the new compose file and the old volume's data. Check the plan against the locked project before capture, then reject or rebuild it with matching volume locks and leases.

Prompt To Fix With AI
This is a comment left during a code review.
Path: backend/internal/project/children/backup/service.go
Line: 506-509

Comment:
**Queued backups use old volumes.** `StartBackup` freezes `p.Volumes` before taking the project lock, but `capture` reloads the project's current files. A compose edit and deploy can finish while the backup is queued or waiting for that lock. If they switch the project to another volume, the backup can succeed with the new compose file and the old volume's data. Check the plan against the locked project before capture, then reject or rebuild it with matching volume locks and leases.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Fix in Codex Fix in Claude Code Fix in Cursor

db.Table("system_backup_policies").Where("s3_destination_id = ? AND s3_enabled = ?", id, true),
db.Table("volume_backups").Where("s3_destination_id = ? AND remote_snapshot_id <> ''", id),
db.Table("volume_backup_policies").Where("s3_destination_id = ? AND s3_enabled = ?", id, true),
db.Table("project_backups").Where("s3_destination_id = ? AND remote_snapshot_id <> ''", id),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Active S3 destinations remain editable. On the first manual backup to a destination, s3_destination_id is set but remote_snapshot_id stays empty until the S3 write finishes. This filter lets the destination change or be deleted during that write. The snapshot can finish in the old bucket while its row points at the changed or deleted destination, leaving the backup unreachable. Count running project backups as references too.

Prompt To Fix With AI
This is a comment left during a code review.
Path: backend/internal/s3/service.go
Line: 262

Comment:
**Active S3 destinations remain editable.** On the first manual backup to a destination, `s3_destination_id` is set but `remote_snapshot_id` stays empty until the S3 write finishes. This filter lets the destination change or be deleted during that write. The snapshot can finish in the old bucket while its row points at the changed or deleted destination, leaving the backup unreachable. Count running project backups as references too.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Fix in Codex Fix in Claude Code Fix in Cursor

Comment on lines +836 to +839
func SelectComposeFiles(ctx context.Context, selection ComposeFileSelection) error {
if slices.Equal(selectedComposeFiles(ctx, selection.GlobalEnvDirectory, selection.ProjectPath), selection.Files) {
return nil
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Imports change Compose overrides. This equality check loses whether COMPOSE_FILE explicitly selected one file. For a source with .env.global setting COMPOSE_FILE=compose.yaml and an unused compose.override.yaml, the bundle records only compose.yaml. On a destination without that global setting, this early return leaves COMPOSE_FILE unset, so Compose now merges the override. The import can succeed with different images, commands, or settings. Preserve the exact loaded file set and whether automatic overrides apply.

Prompt To Fix With AI
This is a comment left during a code review.
Path: backend/pkg/projects/env.go
Line: 836-839

Comment:
**Imports change Compose overrides.** This equality check loses whether `COMPOSE_FILE` explicitly selected one file. For a source with `.env.global` setting `COMPOSE_FILE=compose.yaml` and an unused `compose.override.yaml`, the bundle records only `compose.yaml`. On a destination without that global setting, this early return leaves `COMPOSE_FILE` unset, so Compose now merges the override. The import can succeed with different images, commands, or settings. Preserve the exact loaded file set and whether automatic overrides apply.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Fix in Codex Fix in Claude Code Fix in Cursor

Comment on lines +46 to +49
SELECT 1 FROM api_key_permissions AS reader
WHERE reader.api_key_id = source.api_key_id
AND reader.permission = 'projects:read'
AND COALESCE(reader.environment_id, '') = COALESCE(source.environment_id, '')

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Mixed scopes miss backup grants. The new grant requires matching environment_id values, but permission checks combine global and per-environment grants. A scoped key with global projects:read and volumes:backup for one environment has both prerequisites there, yet gets no projects:backup row. Handle these mixed scopes in both SQLite and Postgres so the migration grants the permission over their shared environments.

Prompt To Fix With AI
This is a comment left during a code review.
Path: backend/internal/database/migrations/00118_project_backups.go
Line: 46-49

Comment:
**Mixed scopes miss backup grants.** The new grant requires matching `environment_id` values, but permission checks combine global and per-environment grants. A scoped key with global `projects:read` and `volumes:backup` for one environment has both prerequisites there, yet gets no `projects:backup` row. Handle these mixed scopes in both SQLite and Postgres so the migration grants the permission over their shared environments.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Fix in Codex Fix in Claude Code Fix in Cursor

Comment on lines +204 to +205
if p.S3DestinationID = strings.TrimSpace(request.S3DestinationID); p.S3DestinationID == "" {
return plan{}, common.Classify(common.ErrBadRequest, errors.New("select an S3 destination for the project backup"))

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Plan fields change after creation. resolvePlan builds p, then fills S3DestinationID, Volumes, and the inspected volume fields afterward. The Go rule requires one keyed literal per value, with known fields set inside it. Resolve and validate these inputs first, then build the inspected entries and final plan with literals.

Rule Used: --- name: go-struct-literals description: Write Go 1.27+ in a declarative, composite-literal style. Every value (structs, slices, maps, configs, errors, servers, test tables, constructor results) is built with one keyed composite literal instead of f... (source)

Prompt To Fix With AI
This is a comment left during a code review.
Path: backend/internal/project/children/backup/service.go
Line: 204-205

Comment:
**Plan fields change after creation.** `resolvePlan` builds `p`, then fills `S3DestinationID`, `Volumes`, and the inspected volume fields afterward. The Go rule requires one keyed literal per value, with known fields set inside it. Resolve and validate these inputs first, then build the inspected entries and final `plan` with literals.

**Rule Used:** --- name: go-struct-literals description: Write Go 1.27+ in a declarative, composite-literal style. Every value (structs, slices, maps, configs, errors, servers, test tables, constructor results) is built with one keyed composite literal instead of f... ([source](https://app.greptile.com/-/switch-tenant?tenant=getarcaneapp&next=%2Fcustom-context%3Fmemory%3D214b40a8-9695-4738-986d-5949b5d65ff1))

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Fix in Codex Fix in Claude Code Fix in Cursor

// projectBundles exports and imports projects with their volume data through the volume operations injected for transfers.
projectBundles struct {
local localTransferEnvironment
activity activity.MessageAppender

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Unused activity dependency. The new projectBundles.activity field is never read, but New and the tests still wire it in. Remove the field, its import, and its wiring so bundle code does not advertise a dependency it never uses.

Prompt To Fix With AI
This is a comment left during a code review.
Path: backend/internal/project/service.go
Line: 208

Comment:
**Unused activity dependency.** The new `projectBundles.activity` field is never read, but `New` and the tests still wire it in. Remove the field, its import, and its wiring so bundle code does not advertise a dependency it never uses.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Fix in Codex Fix in Claude Code Fix in Cursor

neurekadev and others added 3 commits November 18, 2026 18:38
A project whose only active containers were in a restart loop looked
stopped: the export dialog hid the stop switch and the bundle was marked
consistent while those containers could still write their volumes. The
export preview and the consistency check now count running and
restarting containers, the same ones a stop stops.

ComposeVolumeTargetName also takes an options struct instead of four
ordered strings.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A live backup of a project whose only active containers were in a
restart loop was marked consistent. It now uses the same running or
restarting check as export.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
}

// Export packages the project archive and the data of the requested volumes into a bundle upload session.
func (b projectBundles) Export(ctx context.Context, opts ExportBundleOptions) (*types.UploadSession, error) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Bundle code ignores feature layout. projectBundles adds export, import, volume restore, and rollback logic to the parent service.go. AGENTS.md requires substantial features under children/<feature>/ and one job per file. Move this bundle code into a project child and keep the parent focused on connecting it to project operations.

Context Used: AGENTS.md (source)

Prompt To Fix With AI
This is a comment left during a code review.
Path: backend/internal/project/service.go
Line: 1832

Comment:
**Bundle code ignores feature layout.** `projectBundles` adds export, import, volume restore, and rollback logic to the parent `service.go`. `AGENTS.md` requires substantial features under `children/<feature>/` and one job per file. Move this bundle code into a project child and keep the parent focused on connecting it to project operations.

**Context Used:** AGENTS.md ([source](https://github.com/getarcaneapp/arcane/blob/main/AGENTS.md))

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Fix in Codex Fix in Claude Code Fix in Cursor

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant