Skip to content
24 changes: 19 additions & 5 deletions .claude/skills/sc-cli-reference/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -431,9 +431,9 @@ cargo run --bin sc -- remote add <name> sc+git::<git-url> [--allow-private-branc
# `git ls-remote` lists them; the same repo can also
# hold a `--git` mirror of public branches. Anyone
# with push to the repo holds bucket put + delete;
# deleting a ref is not erasure. `remote compact`
# and `serve --store` refuse it in v1; clone
# --filter refused as for any bucket. Untested
# deleting a ref is not erasure. `serve --store`
# refuses it in v1; `remote compact` works (below);
# clone --filter refused as for any bucket. Untested
# against real GitHub
cargo run --bin sc -- remote compact <name> [--dry-run] [--prune-expire 24h] [--allow-short-grace]
# bucket remotes only (#216, ADR-0057): merge small
Expand All @@ -451,7 +451,18 @@ cargo run --bin sc -- remote compact <name> [--dry-run] [--prune-expire 24h] [--
# or fetches running). On a versioned S3
# bucket deletes only add delete markers (add a
# noncurrent-version lifecycle rule). Relieves the
# 65536-pack checkpoint cap (not the ref cap)
# 65536-pack checkpoint cap (not the ref cap).
# On `sc+git::` (ADR-0057 addendum) a Git host has no
# trustworthy object age, so the age is counted from
# the compactor's first sighting of each pack and
# checkpoint (refs/sc/compact/sightings): the first
# run deletes nothing it did not already see, a later
# one at least --prune-expire on may (with a zero
# grace a first sighting is immediately old). Run it
# from ONE host (clocks); a malformed record blocks
# the run (delete that ref to start over). Every
# pusher holds delete there; deleting a ref is not
# erasure
cargo run --bin sc -- serve --http <addr> <home> --store sc+wal://<dir>|sc+s3://<bucket>/<prefix>
# serve a bucket remote over sc+http(s):// (P36c):
# all content lives in the bucket (instance is
Expand All @@ -463,7 +474,10 @@ bash demo/run_git_carrier_demo.sh # sc+git:: proof (P39): a bare Git
# `git clone` sees no refs/sc, keyless clone sees
# `(private, no access)` and no plaintext, keyed
# clone reads all, public main also mirrored as a
# normal Git branch; second leg against sc-forge;
# normal Git branch; `remote compact` twice a grace
# window apart shrinks refs/sc/* (the first run
# deletes nothing) and the clones after it still
# read; second leg against sc-forge;
# no sc-git-bucket-* temp leftovers
bash demo/run_http_remote_demo.sh # sc-native HTTP transport proof (P26): real
# loopback TCP (no shim, unlike ssh://), clone +
Expand Down
25 changes: 19 additions & 6 deletions ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -1372,17 +1372,30 @@ length. Git is the system binary (`SC_GIT` overrides) with a scrubbed environmen
transport allowlist (`https`, `http`, `ssh`, `file`; `ext`, `fd` and `git` forced off), a
URL allowlist and no userinfo on `http(s)` URLs (credentials go through git's credential
helper); a private bare repo in `sc-git-bucket-<pid>-<random>` caches fetched
objects and is removed on `Drop`. v1 scaling limit: every `ls-remote` and push downloads the
objects and is removed on `Drop`. Scaling: every `ls-remote` and push downloads the
whole ref advertisement (the host does not filter it server-side), and a cold read fetches
every `log/` and `checkpoints/` entry; with compaction refused, refs grow by at least three
per push.
every `log/` and `checkpoints/` entry behind the last checkpoint; refs grow by at least three
per push until `sc remote compact` runs.

`sc remote compact` works on a carrier (ADR-0057 addendum). A Git host reports no
trustworthy object age (committer time is writer-controlled), so the compactor keeps its
own: a first-sighting record at `refs/sc/compact/sightings`, one `<first_seen> <oid> <key>`
line per `packs/` and `checkpoints/` key, aged on the compactor's clock and
generation-aware by oid (a key deleted and uploaded again starts over). The record supplies
ages only, decodes strictly (a bad record blocks the run) and is written before any delete
decision uses it. Pack sizes for the merge plan come from each live idx (`offset + 4 +
length`), never from `list_meta`, which would download every value. Deletes are leased
(`Bucket::delete_leased`: one `git push` of `:refs/sc/K` with a
`--force-with-lease=refs/sc/K:<oid>` per key, the oid the one listing showed), and the
manifest is read uncached before every delete call. The first run after an object
appears never deletes it; run compaction from one host.

A private branch goes to an `sc+git::` remote only under the per-remote opt-in
`--allow-private-branches` (`sc remote set <name> --allow-private-branches` for an existing
remote), since its name, recipient ids and sizes become readable by
every reader of the repository and the ciphertext cannot be taken back. `sc remote
compact` (no trustworthy age for a stored pack) and `sc serve --store` (a daemon cannot
answer prompts, and it would bypass the opt-in) refuse an `sc+git::` store in v1.
every reader of the repository and the ciphertext cannot be taken back.
`sc serve --store` (a daemon cannot answer prompts, and it would bypass the opt-in)
refuses an `sc+git::` store in v1.
sc-forge accepts exactly the `refs/sc/` prefix (ADR-0058 addendum). The dependency rule
is amended: `objio` may spawn the system `git`, and only `gitio::bridge` and objio's
git backend spawn it; `gix` stays in `gitio`. Proven by `crates/cli/tests/git_bucket.rs`,
Expand Down
18 changes: 14 additions & 4 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -164,7 +164,7 @@ the code, those win.
| P37c | Merge, cherry-pick and rebase on a partial clone: a tree-recursive engine (`merge_trees`) carries a subtree by id when the sides agree and no protection rule applies, reads only where they differ or a rule intersects, and on a lazy, online clone fills what it asks for by id outside the store lock (a non-lazy clone gets the gap error with `sc backfill <path>`); a carried protected subtree keeps the complete input wrap union; a conflict inside the view is marked and resolved like on a full clone (`sc conflicts`/`sc resolve`/`merge --abort`/`sc commit`; the out-of-view part is grafted from the decided root), while a conflict outside the view, or a result that must be sealed outside it, is refused before any ref, merge-state or working-tree change (fetched objects stay). A subtree identical on all three sides is carried even under a protection rule. Cherry-pick and rebase use the same engine (`replay_commit`): the fold's accumulator may hold absent subtrees, the view is filled before the final write, and a stopped pick or rebase keeps its carried subtrees in the decided root. Mode-only edits now merge and a file/directory collision fails before any write, on full clones too. | [0049](docs/adr/0049-merge-on-partial-clones.md) |
| P37d | Agent sessions on a partial clone: `sc ws fork`/`run`/`harvest`/`abandon` and `sc work` work; a workspace or agent sees the host's sparse view only (`sc work` no longer gets a complete tree there), the view closure is filled (lazy, online) or refused with the gap error before anything is written, a workspace commit grafts the out-of-view subtrees from the base without reading them, and harvest lands through the ADR-0049 merge (any conflict falls back to a `work-<i>` branch as on a full clone) | [0050](docs/adr/0050-sessions-on-partial-clones.md) |
| P38 | Gitea-compatible forge over bare Git and smart HTTP: recorded API subset, scoped auth, protected merges, maintenance and container operations; 24 recorded scenarios and six first-consumer acceptance scenarios pass. Hosted repos have none of the `.sc/` features; SSH and Actions remain outside v1. | [0058](docs/adr/0058-sc-forge-gitea-compatible-git-forge.md) |
| P39 | `sc+git::<git-url>` remotes: the bucket WAL stored as one parentless commit per key at `refs/sc/<key>` in any Git repository (system `git` via `objio::GitBucket`, lease CAS, no `+` refspec), so private branches, protected paths and secrets travel through a Git host (local bare repo, sc-forge; GitHub untested); private-branch push needs per-remote `--allow-private-branches`; `remote compact` and `serve --store` refuse it in v1; sc-forge accepts `refs/sc/` | [0060](docs/adr/0060-git-carried-bucket-remotes.md) |
| P39 | `sc+git::<git-url>` remotes: the bucket WAL stored as one parentless commit per key at `refs/sc/<key>` in any Git repository (system `git` via `objio::GitBucket`, lease CAS, no `+` refspec), so private branches, protected paths and secrets travel through a Git host (local bare repo, sc-forge; GitHub untested); private-branch push needs per-remote `--allow-private-branches`; `remote compact` works on it (first-sighting age record, leased deletes; ADR-0057 addendum) while `serve --store` refuses it in v1; sc-forge accepts `refs/sc/` | [0060](docs/adr/0060-git-carried-bucket-remotes.md) |

## Standing boundaries & gotchas

Expand Down Expand Up @@ -228,8 +228,15 @@ transport-adjacent. The rest, imperatively:
error on it and it never reaches the P18 bridge. Pushing a private branch needs
the per-remote opt-in `--allow-private-branches` (`remote add`/`clone`;
enforced in `repo`, refuses before any upload); protected paths and secrets
need none. `sc remote compact` and `sc serve --store` refuse it (deferred: no
trustworthy pack age, no prompts in a daemon). Every git call goes through
need none. `sc serve --store` refuses it (deferred: no prompts in a daemon).
`sc remote compact` works on it with its own age source (a Git host has no
trustworthy object age): a first-sighting record at
`refs/sc/compact/sightings` (`<secs> <oid> <key>`, generation-aware, strict
decode: a bad record blocks the run, a time over five minutes ahead of the
compactor's clock is refused), leased deletes (`Bucket::delete_leased`), and
an uncached manifest read before every delete call. The first run
deletes nothing it did not already see (zero grace: a first sighting is
immediately old); run it from one host (clocks). Every git call goes through
`objio::GitBucket` with a scrubbed environment; CAS is `--force-with-lease`,
never a `+` refspec, and a rejected push is classified by re-reading the ref.
**Everyone with push to the Git repository holds bucket put and delete** (no
Expand All @@ -255,7 +262,10 @@ transport-adjacent. The rest, imperatively:
not hold. It advances the manifest only through the normal CAS (one no-op
entry plus a checkpoint) and never deletes what the current chain names. Keep `--prune-expire` well above any push's duration (default
24h): a writer stalled longer than that between uploading a pack and its
commit can still lose the pack (the leases follow-on closes it).
commit can still lose the pack (the leases follow-on closes it). On every
backend the manifest is read uncached before every delete call (a busy remote
can defer cleanup to a rerun); on an `sc+git::` remote every pusher also holds
delete.
`--dry-run` writes and deletes nothing but still downloads and decodes every
candidate pack, so on S3 its download egress and read requests match a real run (the uploads,
CAS and deletes it skips are the rest).
Expand Down
7 changes: 4 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -142,9 +142,10 @@ stores the same bucket inside any (SHA-1) Git repository as `refs/sc/*`, so priv
branches, protected paths and secrets travel through a Git host that a plain
`git clone` never sees into (ADR-0060). Pushing a private branch there needs
`--allow-private-branches` (its name and recipient ids become readable by
everyone who can read the repository); compaction and `serve --store` are not
supported on it yet. Proven by `demo/run_git_carrier_demo.sh`; not tested
against GitHub itself.
everyone who can read the repository). `sc remote compact` works on it (run it
from one host; the first run only records when it first saw each object);
`serve --store` is not supported on it yet. Proven by
`demo/run_git_carrier_demo.sh`; not tested against GitHub itself.

## How Phase 1 works

Expand Down
64 changes: 40 additions & 24 deletions ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -703,9 +703,11 @@ across every phase.
sc-forge verified; real GitHub untested); a plain `git clone` never fetches
`refs/sc/*`, and the same repository can also mirror public branches through a
`--git` remote. A private-branch push needs the per-remote opt-in
`--allow-private-branches`; `sc remote compact` and `sc serve --store` refuse
`sc+git::` in v1; sc-forge accepts the `refs/sc/` prefix. No format, wire or
crypto change. Proven by `demo/run_git_carrier_demo.sh`. (ADR-0060.)
`--allow-private-branches`; `sc remote compact` works on `sc+git::` (a
first-sighting age record, leased deletes; ADR-0057 addendum) while
`sc serve --store` refuses it in v1; sc-forge accepts the `refs/sc/` prefix. No
format, wire or crypto change. Proven by `demo/run_git_carrier_demo.sh`.
(ADR-0060.)

## Active

Expand Down Expand Up @@ -1016,27 +1018,41 @@ P28 review follow-ons, the port de-flake, the clippy allow-list) are recorded in
issues a `has_object` round trip per object over S3; batching those probes
into one `refresh()` plus local index lookups (instead of one S3 round
trip per object) is deferred.
- **`sc+git::` follow-ons (P39, ADR-0060).** Deferred rather than built: (1)
`sc remote compact` on an `sc+git::` remote, which needs an age source the
compactor controls (a Git repository gives only writer-controlled committer
time, so `list_meta` reports observation time and v1 refuses the command);
(2) `sc serve --store sc+git::...` (a daemon cannot answer ssh or credential
prompts, and it would sit between clients and the private-branch opt-in); (3)
rollback and rewind of `refs/sc/manifest` by anyone with push access, tracked
under P34 follow-on (6) and ADR-0059 follow-on (2), now reachable by more
people; (4) validation against real GitHub (per-ref and per-push limits,
rulesets, hosts that refuse unknown ref namespaces), untested so far; (5)
batched deletes (one `git push` for many `:refs/sc/K`) and a batched
`has_objects`, since a Git host costs a round trip per probe or delete; (6)
`list_meta` downloads every listed value to read its size; (7) scaling: every
`ls-remote` and push downloads the whole ref advertisement (the host does not
filter it by prefix server-side, a packet trace showed the full list) and a
cold read fetches every `log/` and `checkpoints/` entry, and with compaction
refused (1) refs grow by at least three per push, so a long-lived store slows
down until compaction or a ref-filtering host exists; (8) SHA-256 Git
repositories (v1 pins its cache to SHA-1 and handles SHA-1 ids only); (9) a
download budget for fetches into the temp cache (`git fetch` has no input-size
limit, so an oversized value is written before the read caps refuse it).
- **`sc+git::` follow-ons (P39, ADR-0060).** Built: (1) `sc remote compact` on
an `sc+git::` remote (ADR-0057 addendum). A Git repository gives only
writer-controlled committer time, so the compactor keeps its own age source: a
first-sighting record at `refs/sc/compact/sightings`, generation-aware by oid,
with leased deletes and an uncached manifest check before every destructive
batch; (5a) batched deletes (`Bucket::delete_leased`, one `git push` per chunk
of refs); (6) compaction no longer calls `list_meta` on a carrier (pack sizes
come from the idx; `list_meta` itself still downloads every listed value); (7)
in part: compaction folds small packs and deletes what nothing names, which
bounds the ref growth of at least three per push and the log/checkpoint
entries a cold read fetches, but between runs every `ls-remote` and push still
downloads the whole ref advertisement (the host does not filter it by prefix
server-side, a packet trace showed the full list). Still deferred: (2)
`sc serve --store sc+git::...` (a daemon cannot answer ssh or credential
prompts, and it would sit between clients and the private-branch opt-in); (4)
validation against real GitHub (per-ref and per-push limits, rulesets, hosts
that refuse unknown ref namespaces), untested so far; (9) a download budget for
fetches into the temp cache (`git fetch` has no input-size limit, so an
oversized value is written before the read caps refuse it). Declined, with
reasons: (3) rollback and rewind of `refs/sc/manifest` by anyone with push
access is the cross-cutting signed-ref effort already tracked under P34
follow-on (6) and ADR-0059 follow-on (2), now reachable by more people; a
client-side "highest manifest seq seen" floor was reviewed and judged a false
control; (5b) a batched `has_objects`: the 2 s
freshness window already limits the per-object manifest probe to one network
call per 2 s; (8) SHA-256 Git repositories (v1 pins its cache to SHA-1): GitHub
does not support them and sc-forge is SHA-1, so the value is low and the cost
moderate (the cache's object format would have to be detected before the first
fetch); (S2, from review) the ssh user name that `remote list` prints is not a
credential, and `http(s)` userinfo is refused at validation. Known issue,
recorded and not fixed: a push to `refs/sc/K` fails with "dst refspec matches
more than one" when a decoy branch such as `refs/heads/refs/sc/K` exists on the
host (found while building `serve --store`). Whoever can push to the
repository can plant such a decoy, so it is a writer-only denial of service on
that key; reads compare exact full ref names and are not affected.
- **Serve-side persistent pack cache (P36c follow-on).** A bucket-backed
serve instance re-downloads packs per connection; a content-addressed
on-disk cache in the serve home would make warm instances cheap without
Expand Down
Loading
Loading