Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
8409bfb
feat(forge): accept refs/sc/* for sc+git:: stores
tonibergholm-codento Nov 21, 2026
ca41ce7
feat(objio): GitBucket, a bucket stored as refs/sc/* in any Git repos…
tonibergholm-codento Nov 21, 2026
ee14436
fix(objio): keep family fetches shallow so a hostile host cannot send…
tonibergholm-codento Nov 21, 2026
85cf18d
feat(repo,cli): sc+git:: bucket remotes with a per-remote private-bra…
tonibergholm-codento Nov 21, 2026
f8ffb56
test: run the sc+git:: end-to-end tests against the real GitBucket
tonibergholm-codento Nov 21, 2026
5d0c0bd
docs: ADR-0060 and P39 docs for sc+git:: remotes
tonibergholm-codento Nov 21, 2026
10461d9
demo: sc+git:: carrier demo
tonibergholm-codento Nov 21, 2026
f0a7b9b
fix(objio): harden GitBucket after review of the sc+git:: carrier
tonibergholm-codento Nov 21, 2026
44bac41
fix(repo,cli): redact sc+git:: URLs, close the private gate race, add…
tonibergholm-codento Nov 21, 2026
d2a7a16
docs: record sc+git:: scaling limit and review fixes
tonibergholm-codento Nov 21, 2026
b25a798
fix(objio): pin the git cache to sha1 and never refresh an older cach…
tonibergholm-codento Nov 21, 2026
4900165
docs(adr-0060): state exactly which messages redact carrier URLs
tonibergholm-codento Nov 21, 2026
2bd01df
fix(objio): build cleanly on Windows (unix-only mut and test helpers)
tonibergholm-codento Nov 21, 2026
d9b4281
docs: say the sc+git:: caps bound reads, not downloads
tonibergholm-codento Nov 21, 2026
a89836f
test(forge): generate bucket-value nonces with random_hex (CodeQL)
tonibergholm-codento Nov 21, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 36 additions & 0 deletions .claude/skills/sc-cli-reference/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -407,6 +407,34 @@ cargo run --bin sc -- remote add <name> sc+s3://<bucket>/<prefix> # same, on a
# content stays ciphertext); `clone --filter` is
# refused; signatures, transcripts and drop
# tombstones travel like on any native remote (#229)
cargo run --bin sc -- remote add <name> sc+git::<git-url> [--allow-private-branches]
# bucket remote stored INSIDE a Git repository as
# refs/sc/* (P39, ADR-0060); <git-url> is https://,
# http://, ssh://, file:///abs, scp-form
# (git@host:o/r.git) or an absolute path. `sc+git:`
# is a reserved prefix (only `sc+git::<url>` parses),
# `--git` is an error on it, no password in the URL
# (use git's credential helper / ssh agent; any
# userinfo on an http(s) URL is refused), SC_GIT
# overrides the git binary. `clone sc+git::<url> <dst>
# [--allow-private-branches]` takes the same URL and
# sets the opt-in on `origin`. Pushing a PRIVATE
# branch needs --allow-private-branches (its name,
# recipient ids and sizes become readable by everyone
# who can read the repo, ciphertext stays after a
# revoke); it refuses before any upload without it.
# `sc remote set <name> --allow-private-branches`
# (or --no-allow-private-branches) changes an
# existing remote.
# Protected paths and secrets need no opt-in. A plain
# `git clone` of the repo never fetches refs/sc/*;
# `git ls-remote` lists them; the same repo can also
# hold a `--git` mirror of public branches. Anyone
# with push to the repo holds bucket put + delete;
# deleting a ref is not erasure. `remote compact`
# and `serve --store` refuse it in v1; clone
# --filter refused as for any bucket. Untested
# against real GitHub
cargo run --bin sc -- remote compact <name> [--dry-run] [--prune-expire 24h] [--allow-short-grace]
# bucket remotes only (#216, ADR-0057): merge small
# packs behind one manifest CAS, then delete log
Expand All @@ -429,6 +457,14 @@ cargo run --bin sc -- serve --http <addr> <home> --store sc+wal://<dir>|sc+s3://
# all content lives in the bucket (instance is
# disposable); <home> still needs `.sc/` for tokens,
# the TLS identity and scratch space
bash demo/run_git_carrier_demo.sh # sc+git:: proof (P39): a bare Git repo as the host;
# protected path + secret + private branch pushed
# (private branch refused without the opt-in), plain
# `git clone` sees no refs/sc, keyless clone sees
# `(private, no access)` and no plaintext, keyed
# clone reads all, public main also mirrored as a
# normal Git branch; second leg against sc-forge;
# no sc-git-bucket-* temp leftovers
bash demo/run_http_remote_demo.sh # sc-native HTTP transport proof (P26): real
# loopback TCP (no shim, unlike ssh://), clone +
# push + fetch over sc+http://, a signed ~1 MiB
Expand Down
52 changes: 48 additions & 4 deletions ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ composability held across every phase since. **All three long-term-thesis pillar
now ship** — in-RAM worktrees (P1), committed secrets (P2), and per-file
permissions (P7, encrypted paths; per-branch in P34) — atop a full collaborative
VCS: three-way merge and history editing, packfiles + GC, ssh:// / sc+http(s):// /
bucket (sc+wal://, sc+s3://) / Git network transports, sparse and partial clone
bucket (sc+wal://, sc+s3://, sc+git::) / Git network transports, sparse and partial clone
(with lazy fetch, merge and agent sessions on a partial clone), signed commits,
intent-deep provenance via sealed agent-session transcripts, and a read-only
native desktop browser. The phase-by-phase build log is in
Expand Down Expand Up @@ -82,8 +82,9 @@ src-control/
│ │ v2 dual-key identities (scl-crypto; the only crate linking RustCrypto)
│ ├── tlsio/ rustls/rcgen wrappers for sc+https:// (P32); a leaf, the only crate
│ │ linking rustls
│ ├── objio/ `Bucket` trait + directory and S3 backends for sc+wal:// / sc+s3:// (P36);
│ │ a leaf, the only crate linking the S3 SDK
│ ├── objio/ `Bucket` trait + directory, S3 and Git backends for sc+wal:// / sc+s3:// /
│ │ sc+git:: (P36, P39); a leaf, the only crate linking the S3 SDK; its Git
│ │ backend spawns the system `git` (never gix)
│ ├── repo/ durable .sc/ repo: refs/branches/working tree, three-way merge, history
│ │ editing (cherry-pick/rebase/amend/undo), remotes + wire transport (ssh://,
│ │ sc+http(s)://, bucket WAL), protection + secret lifecycle, GC, sparse &
Expand Down Expand Up @@ -220,7 +221,8 @@ lazily-built bare `mirror.git` beside the marks map is the transport). All of it
keeps `gix` quarantined in `gitio`. This is distinct from sc's **own** network
transport: `repo` speaks a framed wire protocol over ssh:// (P12), sc+http://
(P26) and sc+https:// (P32) that mirrors its `Transport` trait, and treats a
bucket as a remote (P36) — no `gix` involved (see the appendix).
bucket as a remote (P36), which P39 can store inside a Git repository (`sc+git::`,
through the system `git` binary, not `gix`) — no `gix` involved (see the appendix).

## Cryptography and key management

Expand Down Expand Up @@ -1344,3 +1346,45 @@ out-of-view subtrees from the base without reading them; and harvest lands
through the ADR-0049 merge, with any conflict falling back to a `work-<i>`
branch as on a full clone. `sc backfill --all` remains the way to a genuine full
clone. See ADR-0050.

## Phase 39 — Git-carried bucket remotes: sc+git:: (built)

The bucket WAL (P36) stored inside any Git repository, so a Git host can carry
private branches, protected paths and secrets that the Git bridge cannot.
`sc+git::<git-url>` opens `objio::GitBucket` behind the same `Bucket` trait, so
`BucketTransport`, the log, the manifest and every format stay unchanged: no wire
change, no `PROTOCOL_VERSION` bump, no new crypto. Key `K` is the ref `refs/sc/K`;
its value is a parentless commit whose tree holds the bytes as chunk blobs
`c0000`, `c0001`, ... (at most 32 MiB each, 2 GiB per value), and the commit id is
the CAS tag. A default `git clone` or `git fetch` never takes `refs/sc/*`, and the
P18 bridge touches only `refs/heads/*`, so one repository can hold the sc store and
a plain Git mirror of the public branches.

`put_new` and `put_if_tag` push with `--force-with-lease` (an empty lease for
create-only), never a `+` refspec, so `receive-pack` checks the sent old value under
its ref lock and exactly one racer wins. A rejected push is classified by re-reading
the exact ref, so a hook refusal is an error and never a lost race. Reads keep a 2 s
freshness window for a tag the instance itself saw or wrote (dropped when a write of
that key does not land), serve create-only values (decided by key layout: `log/`,
`checkpoints/`, `packs/*.pack|idx`) from the local cache, and fetch the `log/`, `checkpoints/` and `packs/*.idx` families
in one shallow `git fetch`, which bounds a cold clone's round trips independent of log
length. Git is the system binary (`SC_GIT` overrides) with a scrubbed environment, a
transport allowlist (`https`, `http`, `ssh`, `file`; `ext`, `fd` and `git` forced off), a
URL allowlist and no userinfo on `http(s)` URLs (credentials go through git's credential
helper); a private bare repo in `sc-git-bucket-<pid>-<random>` caches fetched
objects and is removed on `Drop`. v1 scaling limit: every `ls-remote` and push downloads the
whole ref advertisement (the host does not filter it server-side), and a cold read fetches
every `log/` and `checkpoints/` entry; with compaction refused, refs grow by at least three
per push.

A private branch goes to an `sc+git::` remote only under the per-remote opt-in
`--allow-private-branches` (`sc remote set <name> --allow-private-branches` for an existing
remote), since its name, recipient ids and sizes become readable by
every reader of the repository and the ciphertext cannot be taken back. `sc remote
compact` (no trustworthy age for a stored pack) and `sc serve --store` (a daemon cannot
answer prompts, and it would bypass the opt-in) refuse an `sc+git::` store in v1.
sc-forge accepts exactly the `refs/sc/` prefix (ADR-0058 addendum). The dependency rule
is amended: `objio` may spawn the system `git`, and only `gitio::bridge` and objio's
git backend spawn it; `gix` stays in `gitio`. Proven by `crates/cli/tests/git_bucket.rs`,
the forge end-to-end tests and `demo/run_git_carrier_demo.sh`. Not tested against real
GitHub. See ADR-0060.
27 changes: 25 additions & 2 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,11 @@ yourself reaching for it elsewhere, add a function to `crypto` instead.
**rustls/rcgen must stay quarantined in `tlsio`** — if you find yourself
reaching for TLS elsewhere, add a function to `tlsio` instead. **Object-store
SDKs must stay quarantined in `objio`** — if you find yourself reaching for
S3 elsewhere, add a function to `objio` instead. **`repo` must not depend on
S3 elsewhere, add a function to `objio` instead. **`objio` also holds the
`sc+git::` backend (P39), which may spawn the system `git` binary (never `gix`);
the system `git` binary is spawned only by `gitio::bridge` and objio's git
backend** (`repo` must not depend on `gitio`, and `repo` picks the bucket
backend). **`repo` must not depend on
`gitio`** — `cli` links both and passes imported snapshots down; `repo` stays
Git-agnostic. **`forge` (the `sc-forge` Git forge, P38, unix-only) is a third top-level adapter
and may link only `{crypto, core}`** — never `repo`, `vfs` or `gitio` (hosted
Expand Down Expand Up @@ -110,7 +114,7 @@ the project tree if desired.

## Capability map (what's built, by phase)

All 36 phases, P37a, P37b-i, P37b-ii, P37c, P37d and P38 are built and tested. One line of current fact per phase; the
All 36 phases, P37a, P37b-i, P37b-ii, P37c, P37d, P38 and P39 are built and tested. One line of current fact per phase; the
authoritative rationale and full semantics live in the linked ADR, the design
in `ARCHITECTURE.md`. The old per-phase narrative log this table replaced is
archived verbatim at `docs/archive/claude-md-phase-log-2026-07.md` — do not
Expand Down Expand Up @@ -160,6 +164,7 @@ the code, those win.
| P37c | Merge, cherry-pick and rebase on a partial clone: a tree-recursive engine (`merge_trees`) carries a subtree by id when the sides agree and no protection rule applies, reads only where they differ or a rule intersects, and on a lazy, online clone fills what it asks for by id outside the store lock (a non-lazy clone gets the gap error with `sc backfill <path>`); a carried protected subtree keeps the complete input wrap union; a conflict inside the view is marked and resolved like on a full clone (`sc conflicts`/`sc resolve`/`merge --abort`/`sc commit`; the out-of-view part is grafted from the decided root), while a conflict outside the view, or a result that must be sealed outside it, is refused before any ref, merge-state or working-tree change (fetched objects stay). A subtree identical on all three sides is carried even under a protection rule. Cherry-pick and rebase use the same engine (`replay_commit`): the fold's accumulator may hold absent subtrees, the view is filled before the final write, and a stopped pick or rebase keeps its carried subtrees in the decided root. Mode-only edits now merge and a file/directory collision fails before any write, on full clones too. | [0049](docs/adr/0049-merge-on-partial-clones.md) |
| P37d | Agent sessions on a partial clone: `sc ws fork`/`run`/`harvest`/`abandon` and `sc work` work; a workspace or agent sees the host's sparse view only (`sc work` no longer gets a complete tree there), the view closure is filled (lazy, online) or refused with the gap error before anything is written, a workspace commit grafts the out-of-view subtrees from the base without reading them, and harvest lands through the ADR-0049 merge (any conflict falls back to a `work-<i>` branch as on a full clone) | [0050](docs/adr/0050-sessions-on-partial-clones.md) |
| P38 | Gitea-compatible forge over bare Git and smart HTTP: recorded API subset, scoped auth, protected merges, maintenance and container operations; 24 recorded scenarios and six first-consumer acceptance scenarios pass. Hosted repos have none of the `.sc/` features; SSH and Actions remain outside v1. | [0058](docs/adr/0058-sc-forge-gitea-compatible-git-forge.md) |
| P39 | `sc+git::<git-url>` remotes: the bucket WAL stored as one parentless commit per key at `refs/sc/<key>` in any Git repository (system `git` via `objio::GitBucket`, lease CAS, no `+` refspec), so private branches, protected paths and secrets travel through a Git host (local bare repo, sc-forge; GitHub untested); private-branch push needs per-remote `--allow-private-branches`; `remote compact` and `serve --store` refuse it in v1; sc-forge accepts `refs/sc/` | [0060](docs/adr/0060-git-carried-bucket-remotes.md) |

## Standing boundaries & gotchas

Expand Down Expand Up @@ -216,6 +221,24 @@ transport-adjacent. The rest, imperatively:
- **Bucket remotes hold public content plaintext at rest** — bucket ACL is
the perimeter (sealed content stays ciphertext, unchanged); partial-clone
`filter` against bucket remotes is refused.
- **`sc+git::<git-url>` (P39, ADR-0060) stores the bucket WAL as `refs/sc/*` in a
Git repository.** The prefix `sc+git:` is reserved: only `sc+git::<inner>`
parses (inner: `https://`, `http://`, `ssh://`, `file:///abs`, scp-form or an
absolute path; no password in the URL, no `git://`, no `::`), `--git` is an
error on it and it never reaches the P18 bridge. Pushing a private branch needs
the per-remote opt-in `--allow-private-branches` (`remote add`/`clone`;
enforced in `repo`, refuses before any upload); protected paths and secrets
need none. `sc remote compact` and `sc serve --store` refuse it (deferred: no
trustworthy pack age, no prompts in a daemon). Every git call goes through
`objio::GitBucket` with a scrubbed environment; CAS is `--force-with-lease`,
never a `+` refspec, and a rejected push is classified by re-reading the ref.
**Everyone with push to the Git repository holds bucket put and delete** (no
versioning; wider than S3), branch protection does not cover `refs/sc/*`, and
rewind stays the tracked follow-on. **Deleting a ref is not erasure** (hosts
and forks may keep objects by id). Public content is plaintext, and private
branch names, recipient ids and sizes are readable by every reader of the
repository. A killed process can leave `sc-git-bucket-<pid>-<random>` in the
temp dir. Not tested against real GitHub.
- **`DirBucket` (`sc+wal://<dir>`) is a single-host, single-owner backend (#264).**
Its CAS takes an OS advisory file lock (`File::lock`: `flock` on unix,
`LockFileEx` on Windows) on `<root>/.cas-lock`, a file that is never removed and
Expand Down
22 changes: 16 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,8 +64,9 @@ crates/
Ed25519 signing (the ONLY crate that links RustCrypto)
tlsio/ TLS for the sc+https:// transport (the ONLY crate that links
rustls/rcgen; a leaf — no workspace deps)
objio/ object-store buckets for sc+wal:// and sc+s3:// remotes (the ONLY crate
that links the S3 SDK; a leaf — no workspace deps)
objio/ object-store buckets for sc+wal://, sc+s3:// and sc+git:: remotes (the
ONLY crate that links the S3 SDK; its Git backend spawns the system
`git`; a leaf — no workspace deps)
repo/ durable .sc repo — refs, branches, working tree, merge, history
editing, remotes, protection, GC, sparse/partial clone, transcripts,
transport (ssh, sc+http(s), bucket WAL)
Expand Down Expand Up @@ -136,6 +137,14 @@ Bucket remotes take `sc remote add <name> sc+wal://<dir>` or
(ADR-0057), and `sc serve --http 127.0.0.1:8730 --store <url> <serve-home>`
serves a bucket from a disposable instance (`--stdio` also works;
`sc serve --help`).
`sc remote add <name> sc+git::<git-url>` (or `sc clone sc+git::<git-url> <dst>`)
stores the same bucket inside any (SHA-1) Git repository as `refs/sc/*`, so private
branches, protected paths and secrets travel through a Git host that a plain
`git clone` never sees into (ADR-0060). Pushing a private branch there needs
`--allow-private-branches` (its name and recipient ids become readable by
everyone who can read the repository); compaction and `serve --store` are not
supported on it yet. Proven by `demo/run_git_carrier_demo.sh`; not tested
against GitHub itself.

## How Phase 1 works

Expand Down Expand Up @@ -168,11 +177,12 @@ provenance (droppable through signed tombstones), sparse and partial checkouts (
cherry-pick, rebase and agent sessions on a partial clone), and local,
ssh-native, sc-native HTTP/TLS (bearer tokens with expiry and rotation,
listener resource limits, TOFU pinning managed by `sc tls`, or CA validation
via `SC_HTTPS_CA_FILE`/`SC_HTTPS_CA=system`), bucket (`sc+wal://`, `sc+s3://`),
and Git (including hosted GitHub) network transports, plus private branches
via `SC_HTTPS_CA_FILE`/`SC_HTTPS_CA=system`), bucket (`sc+wal://`, `sc+s3://`,
`sc+git::` inside a Git repository), and Git (including hosted GitHub) network
transports, plus private branches
sealed to a recipient set until publish and a read-only Tauri desktop
browser. All 36 phases and P37a-37d are implemented and tested; `demo/` holds
28 independent end-to-end proofs covering most major capabilities. Remaining
browser. All 36 phases, P37a-37d and P39 are implemented and tested; `demo/` holds
29 independent end-to-end proofs covering most major capabilities. Remaining
follow-ons (richer signer trust models, per-path token ACLs, transcript
rewrap/grant/revoke and similar items) are tracked in [ROADMAP.md](ROADMAP.md)'s Deferred section and as open
GitHub issues rather than being open MVP scope.
Loading
Loading