Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 28 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -246,11 +246,34 @@ Commit multiple records as one batch intent:

```bash
cfg commit --bulk-from batch.json -m "switch planner routing"
cfg commit --bulk-from batch.json --dry-run # preview every record's delta, write nothing
```

Bulk commit preflights the whole batch before writing. If any target has
un-adopted drift, is missing, duplicates another target, or trips the secret
policy, cfgit applies none of the batch.
policy, cfgit applies none of the batch. `--dry-run` previews the per-record delta
(`would_commit` / `noop` / `changed_outside_cfgit`) for the whole batch without writing —
run it before any collection-scale replace.

### Snapshot and restore a whole collection

For "back it up, replace many rows, roll the whole thing back," `cfg export` writes a portable,
re-importable snapshot of the current live documents (each stamped with its cfgit head seq/oid),
and `cfg import --from` writes those documents back through the drift-guarded bulk-commit path —
so the restore is recorded in history like any other change:

```bash
cfg export --out backup.json # snapshot every collection to a file
cfg export modelgarden_models:openai/gpt-4o-mini # or one record, to stdout
# ... do a risky bulk change ...
cfg import --from backup.json --dry-run # preview the restore
cfg import --from backup.json -m "restore from backup"
```

This is the file-based backup artifact (portable, out-of-tool, diffable). For an in-tool
rollback with no file, use `cfg tag` + `cfg restore --tag` / `cfg restore --as-of <date>`, which
version the whole system inside cfgit. Both record the restore in history; export/import also
gives you a standalone file to keep or share.

Draft changes on a branch before mutating runtime:

Expand Down Expand Up @@ -324,12 +347,15 @@ cfg init
cfg doctor [record]
cfg doctor --status
cfg import --all -m "initial import"
cfg export [record] [--out file.json]
cfg import --from <export.json> [--dry-run] -m "message"
cfg status [record]
cfg diff <record> [from] [to]
cfg impact <record> [from] [to]
cfg commit <record> --from <file.json> -m "message"
cfg commit <record> --from <file.json> --dry-run
cfg commit --bulk-from <batch.json> -m "message"
cfg commit --bulk-from <batch.json> --dry-run
cfg set <record> field=value nested.field=value -m "message"
cfg edit <record> -m "message"
cfg branch list
Expand Down Expand Up @@ -464,6 +490,7 @@ Tools include:
- `cfg_commit`
- `cfg_bulk_commit`
- `cfg_set`
- `cfg_export`
- `cfg_branch_list`
- `cfg_branch_create`
- `cfg_branch_delete`
Expand Down
34 changes: 33 additions & 1 deletion docs/USAGE.md
Original file line number Diff line number Diff line change
Expand Up @@ -109,7 +109,39 @@ cfg commit --bulk-from batch.json -m "switch planner routing"

Bulk commit preflights every target before writing. If any record has un-adopted
drift, is missing, duplicates another target, or trips the secret policy, no
record in the batch is applied.
record in the batch is applied. Add `--dry-run` to preview the per-record delta for the whole
batch (`would_commit` / `noop` / `changed_outside_cfgit`) without writing — run it before any
collection-scale replace:

```bash
cfg commit --bulk-from batch.json --dry-run
```

## Snapshot and restore a collection

`cfg export` writes a portable, re-importable snapshot of the current live documents, each
stamped with its cfgit head seq/oid. `cfg import --from` writes those documents back through the
drift-guarded bulk-commit path, so the restore is recorded in history like any other change.
Together they make "back it up, do the bulk change, roll it back" a fully in-cfgit workflow:

```bash
cfg export --out backup.json # snapshot every collection to a file
cfg export modelgarden_models:openai/gpt-4o-mini # or one record, to stdout
# ... risky bulk change ...
cfg import --from backup.json --dry-run # preview the restore
cfg import --from backup.json -m "restore from backup"
```

For an in-tool rollback without a file, use `cfg tag` + `cfg restore --tag` (or
`cfg restore --as-of <date>`) instead — see [Tags](#tags) and [Restore](#restore). Both record
the restore in history; export/import additionally gives you a standalone file to keep or share.

`cfg export` warns (without stopping) when a snapshot is large by control-plane standards — a
strong hint that the config points at a data-plane collection (events, user content, jobs),
which cfgit is not designed to version. If you cancel an `import --from` partway, cfgit reports
`partial` with `cancelled: true` listing what committed and what is still pending; each record
is applied atomically, so rerunning the same command resumes (already-applied records are
no-ops).

If the live record changed after cfgit last recorded it, commit returns
`changed_outside_cfgit` and does not apply your document. Inspect the drift first:
Expand Down
5 changes: 4 additions & 1 deletion skills/cfgit/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,8 @@ Use cfgit as the safety layer around a live datastore. The app still reads and w
(returns `would_commit` with the delta, or `changed_outside_cfgit`/`noop`; writes nothing).
- Every outcome now carries a top-level `next` block (why + remedy + copy-paste `commands`);
on a refusal, follow `next.commands` rather than guessing.
- For a coupled multi-record change, write a batch JSON file and run `cfg commit --bulk-from <file> -m "<message>" --json`. The batch file can be `[{"record":"collection:id","doc":{...}}]` or `{"collection:id": {...}}`.
- For a coupled multi-record change, write a batch JSON file and run `cfg commit --bulk-from <file> -m "<message>" --json`. The batch file can be `[{"record":"collection:id","doc":{...}}]` or `{"collection:id": {...}}`. Preview a collection-scale batch first with `cfg commit --bulk-from <file> --dry-run --json` — it reports each record's `would_commit`/`noop`/`changed_outside_cfgit` and writes nothing.
- Before a risky bulk change, snapshot the current live docs with `cfg export --out backup.json --json` (a portable, re-importable artifact stamped with head seq/oid). To roll back, `cfg import --from backup.json -m "<message>" --json` writes them back through the drift-guarded path (preview with `--dry-run`). For an in-tool rollback with no file, use `cfg tag` + `cfg restore --tag` instead.
- For a draft branch, use `cfg --branch <branch> commit <record> --from <file> -m "<message>" --json` or `cfg --branch <branch> commit --bulk-from <file> -m "<message>" --json`. This writes cfgit refs only; runtime is unchanged.
- If commit returns `changed_outside_cfgit`, stop and inspect drift.
- If bulk commit returns `blocked`, no record was applied; inspect `failed`. If it returns `partial`, some records were applied before a race/failure; inspect `results`, `failed`, and `pending` before continuing.
Expand All @@ -61,6 +62,7 @@ Use cfgit as the safety layer around a live datastore. The app still reads and w
4. Reconcile drift.
- `cfg diff <record> =HEAD =live --json`
- `cfg adopt <record> -m "<message>" --json`
- If `cfg doctor --status` reports a high drift ratio (many records `changed_outside_cfgit`), baseline the whole collection once with `cfg adopt --all -m "<message>" --json` so later edits and restore work without a per-record adopt.

5. Restore.
- Single record: `cfg restore <record> <ref> -m "<message>" --json`
Expand All @@ -85,6 +87,7 @@ If the cfgit MCP server is available, prefer its tools over shelling out:
- `cfg_impact`
- `cfg_commit`
- `cfg_bulk_commit`
- `cfg_export`
- `cfg_branch_list`
- `cfg_branch_create`
- `cfg_branch_delete`
Expand Down
45 changes: 40 additions & 5 deletions src/cfg/cli/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -145,8 +145,15 @@ def _parser() -> argparse.ArgumentParser:
p_import = sub.add_parser("import")
p_import.add_argument("record", nargs="?")
p_import.add_argument("--all", action="store_true")
p_import.add_argument("--from", dest="from_file",
help="restore documents from a `cfg export` file (writes via the drift-guarded bulk-commit path)")
p_import.add_argument("-m", "--message", default="initial import")
p_import.add_argument("--allow-secret", action="store_true")
p_import.add_argument("--dry-run", action="store_true", help="with --from: preview the bulk write without applying")

p_export = sub.add_parser("export", help="dump live documents to a portable, re-importable JSON snapshot")
p_export.add_argument("record", nargs="?", help="one collection:id; omit to export every configured collection")
p_export.add_argument("--out", help="write the snapshot to this file instead of stdout")

p_doctor = sub.add_parser("doctor")
p_doctor.add_argument("record", nargs="?")
Expand Down Expand Up @@ -191,7 +198,7 @@ def _parser() -> argparse.ArgumentParser:
p_commit.add_argument(
"--dry-run",
action="store_true",
help="preview the field-level delta vs live and exit without writing (main-branch, single record)",
help="preview the delta and exit without writing; works with --from (single) and --bulk-from (per-record)",
)

p_set = sub.add_parser("set", help="edit scalar fields in place; routes through commit (drift-guarded)")
Expand Down Expand Up @@ -321,15 +328,37 @@ def _dispatch(engine: Engine, args: argparse.Namespace) -> tuple[Any, int]:
raise ValueError(f"unknown PR command: {args.pr_cmd}")

if args.cmd == "import":
if getattr(args, "from_file", None):
from cfg.interfaces.actions import import_from_file

return import_from_file(
engine,
_load_json_any(args.from_file),
message=args.message if args.message != "initial import" else "import from export file",
allow_secret=args.allow_secret,
dry_run=args.dry_run,
)
if not args.all and not args.record:
raise ValueError("import needs --all or a record")
raise ValueError("import needs --all, a record, or --from <export.json>")
result = engine.import_records(
_parse_record(args.record) if args.record else None,
message=args.message,
allow_secret=args.allow_secret,
)
return result, EXIT_OK

if args.cmd == "export":
from cfg.interfaces.actions import export_records as _export_action

report, _ = _export_action(engine, args.record if args.record else None)
warning = report.pop("warning", None)
if warning:
print(f"⚠ {warning}", file=sys.stderr)
if getattr(args, "out", None):
Path(args.out).write_text(json.dumps(_to_json(report), indent=2), encoding="utf-8")
return {"state": "exported", "out": args.out, "count": report["count"]}, EXIT_OK
return report, EXIT_OK

if args.cmd == "status":
rows = engine.status(_parse_record(args.record) if args.record else None)
code = EXIT_DIRTY if any(r.state == "changed_outside_cfgit" for r in rows) else EXIT_OK
Expand Down Expand Up @@ -383,11 +412,15 @@ def _dispatch(engine: Engine, args: argparse.Namespace) -> tuple[Any, int]:
if args.bulk_from_file:
if args.record or args.from_file:
raise ValueError("bulk commit uses --bulk-from without record or --from")
if not args.message:
raise ValueError("bulk commit needs -m/--message")
from cfg.interfaces.actions import bulk_commit_exit_code, parse_bulk_commit_items
from cfg.interfaces.actions import bulk_commit_exit_code, bulk_commit_preview, parse_bulk_commit_items

items = parse_bulk_commit_items(_load_json_any(args.bulk_from_file))
if args.dry_run:
if branch != engine.config.branches.default_branch:
raise ValueError("--dry-run is only supported on the main-branch commit path")
return bulk_commit_preview(engine, items), EXIT_OK
if not args.message:
raise ValueError("bulk commit needs -m/--message")
if branch != engine.config.branches.default_branch:
result = engine.branch_commit_many(
branch,
Expand Down Expand Up @@ -674,6 +707,8 @@ def _format_status_report(r: dict[str, Any]) -> str:
]
for warning in r.get("warnings") or []:
lines.append(f"⚠ {warning}")
for nudge in r.get("nudges") or []:
lines.append(f"→ {nudge}")
return "\n".join(lines)


Expand Down
38 changes: 38 additions & 0 deletions src/cfg/core/engine.py
Original file line number Diff line number Diff line change
Expand Up @@ -493,6 +493,32 @@ def import_records(
results.append({"collection": item.collection, "record_id": item.record_id, "state": "imported", "seq": result.seq, "oid": result.oid})
return results

def export_records(self, ref: RecordRef | None = None) -> dict[str, Any]:
"""Read-only snapshot of live documents into a portable, re-importable artifact.

With a ref, exports that one record; without, exports every configured collection's live
records. Each item carries its live doc plus the current cfgit head seq/oid (or null if
untracked), so the snapshot records exactly which version it represents. Writes nothing.
"""
refs = [ref] if ref else self._all_refs(include_history=False)
items: list[dict[str, Any]] = []
for item in sorted(refs, key=lambda r: (r.collection, r.record_id)):
live = self.adapter.get_record(item.collection, item.record_id)
if live is None:
continue
head = self.adapter.get_head(item.collection, item.record_id)
items.append(
{
"record": f"{item.collection}:{item.record_id}",
"collection": item.collection,
"record_id": item.record_id,
"head_seq": head.get("seq") if head else None,
"head_oid": head.get("oid") if head else None,
"doc": live,
}
)
return {"version": 1, "kind": "cfgit-export", "count": len(items), "items": items}

def doctor(self, ref: RecordRef | None = None, *, large_field_bytes: int = 20000) -> dict[str, Any]:
"""Read-only preflight. Walks live records and reports what would trip an
import/commit BEFORE anything is written: secret-deny matches (grouped by
Expand Down Expand Up @@ -748,6 +774,18 @@ def commit_many(
expected_live_oid=plan["expected_live"],
make_head=True,
)
except KeyboardInterrupt:
# A cancel between per-record writes: each apply() is atomic, so already-committed
# records are durable. Return a clean partial report (what landed, what did not)
# instead of dying silently and leaving the operator guessing about DB state.
pending = [_plan_result(p) for p in plans[offset:]]
return {
"state": "partial",
"cancelled": True,
"results": results,
"failed": [],
"pending": pending,
}
except Exception as exc:
failed = [
{
Expand Down
2 changes: 1 addition & 1 deletion src/cfg/core/remedy.py
Original file line number Diff line number Diff line change
Expand Up @@ -102,7 +102,7 @@ def to_json(self) -> dict[str, Any]:
commands=("cfg commit {record} --from <file> -m '<why>'",),
),
"dry_run": Next(
why="Dry run: this is what a restore would change. Nothing was written.",
why="Dry run: this is what would change. Nothing was written.",
remedy="Rerun without --dry-run to apply.",
commands=(),
),
Expand Down
Loading
Loading