Repository navigation
feat: carry private branches, protected files and secrets through any Git host (sc+git::) - #369
Conversation
The pre-receive policy now accepts create, update (including non-fast-forward) and delete for names under exactly `refs/sc/`, so a repository hosted on sc-forge can carry a native sc bucket store. `refs/sc`, `refs/scx/x` and every other namespace stay refused; heads and tags are unchanged. End-to-end tests drive the real receive path over smart HTTP with plain git: create with an empty lease, CAS update without `+`, stale lease rejected, exact-refspec fetch, delete, two racing CAS pushes with one winner, a hand-built receive-pack request that proves the server checks the old value under the ref lock, and a maintenance pass that keeps the live values and prunes the replaced one. ADR-0058 addendum, FORGE-OPERATIONS.md and THREAT-MODEL.md describe the namespace: writable by every repository writer, outside branch protection, advertised to readers, bounded by receive.maxInputSize.
…itory Adds the objio backend for the sc+git:: carrier (P39). Each bucket key lives at refs/sc/<key> as a parentless commit whose tree holds the value as chunk blobs (32 MiB each, 2 GiB per value). Create-only writes push with an empty lease and CAS writes with the expected oid as the lease, never with a + refspec; every rejection is classified by re-reading the exact ref with ls-remote, so a hook refusal is an error and never a lost race. The backend drives the system git binary (SC_GIT overrides), with a scrubbed GIT_* environment, a protocol allowlist, no hooks or signing, a fixed carrier identity and a private temp cache repository removed on drop. Cold reads of log/, checkpoints/ and packs/*.idx fetch the whole family in one call. Tests cover the contract suite, racing CAS and create, stale and malformed tags, a rejecting pre-receive hook, decoy refs, chunking, corrupt value shapes, URL and key validation, hostile user git config, cache cleanup and a counting git shim that bounds network calls independent of entry count.
… history A glob fetch of a key family now uses --depth=1 like exact fetches. A shallow glob that matches nothing exits 1, so a failed family fetch lists the family once: empty means absent, anything else is a real failure.
…nch opt-in Route `sc+git::<git-url>` to the bucket transport (GitBucket stores the WAL as refs/sc/* in any Git repository). `sc+git:` is a reserved prefix: only the exact `sc+git::<inner>` form parses, and a near miss is refused instead of being saved or read as a local path or sent to the P18 Git bridge. - one `is_bucket_url` replaces the duplicated scheme checks - `remote add --git` and `clone --git` refuse sc+git:: - per-remote `allow_private_branches` in .sc/config (absent = false, omitted on write while false); `remote add`/`clone --allow-private-branches` set it - `Repo::push` refuses a private branch on a gated remote before any I/O, naming the flag and what becomes readable - `remote add`/`clone` print a one-line exposure notice - `remote compact` and `serve --store` refuse sc+git:: (deferred) - bucket errors are escaped for the terminal and no longer expose the raw inner error as an anyhow source - end-to-end tests over a bare Git repo are #[ignore]d until the real GitBucket lands
ADR-0060 records the refs/sc/* mapping, lease CAS, private-branch opt-in, deferred compaction and serve --store, and the dependency-rule amendment. CLAUDE.md, ARCHITECTURE, THREAT-MODEL, ROADMAP, README and the sc-cli-reference skill carry the capability row, boundaries and follow-ons.
Self-checking proof for P39: a bare Git repo and a real sc-forge carry a protected path, a secret and a private branch as refs/sc/*. Covers the private-branch opt-in gate, a plain git clone that sees no sc store, keyless and keyed sc clones, the Git bridge mirroring public main in the same repo, and zero sc-git-bucket-* residue across two runs.
- Drop a key's cached tag after any write that does not land, so the 2 s freshness window cannot answer Unchanged for a tag that just lost a CAS. - Decide create-only by key layout (log/, checkpoints/, packs/*.pack|idx), never by the writer-controlled commit message. - Treat a refused push as success when the remote already holds our new commit (lost acknowledgement); classification is a pure function. - Require a single-key ref to point directly at a commit; no tag peeling. - Force protocol.ext/fd/git.allow=never and fetch.fsckObjects=true. - URL validation: no userinfo on http(s), no leading '-' in ssh/scp user, restricted ssh:// host charset. - Cap every non-.pack value at 256 MiB, checked before reading. - Check family ref names before they enter state; match GIT_* env names case-insensitively on Windows. - Export redact; note SIGINT in the module docs. - Tests: counting shim now covers the writer too (the zero-call assertion was vacuous).
… remote set - Never print a raw sc+git URL: parse errors, the private-branch gate, the --git bails and "added remote" show it with userinfo redacted. - Push reads the branch and tip once, gates on that tip and pushes it. - Compaction is refused in BucketTransport itself for sc+git:: remotes, not only in the CLI. - The reserved sc+git: prefix is case-insensitive; only exact lowercase sc+git::<inner> parses. - Add `sc remote set <name> --allow-private-branches` and `--no-allow-private-branches`; the gate message names it. - CLI tests assert the partial-clone refusal text and that the e2e run leaves no sc-git-bucket-* behind in its TMPDIR.
ADR-0060, ARCHITECTURE, THREAT-MODEL, ROADMAP and the sc-cli-reference skill: lost-ack and layout-only create-only rules, userinfo refusal on http(s), forced protocol denials, 256 MiB cap for non-pack values, and `sc remote set`. Record as a v1 limit that every ls-remote and push downloads the whole ref advertisement (the host does not filter by prefix) and a cold read fetches every log and checkpoint entry, with refs growing by at least three per push while compaction is refused.
…ed value A user default of init.defaultObjectFormat=sha256 made the cache sha256, so every push to a sha1 carrier failed. Confirming a newer tag also refreshed the cache entry of an older value, which could then be answered as unchanged. SHA-256 carriers are a documented v1 limit.
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars.
|
Codex Review Summary: ✅ CompletedThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 2bd01df667
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
CodeQL flags a literal used as a nonce. The forge refs/sc tests now draw it from scl_crypto::random_hex, as the repo does for test passwords.
tonibergholm
left a comment
There was a problem hiding this comment.
Approved after agent reviews (Fable, Mythos, Astra, code-reviewer, sol-loop) and green CI on a89836f.
Theo asked for three things in a Git host: files only some teammates can read, branches only some teammates can see, and built-in secrets. sc already has all three: protected paths (P7), private branches (P34) and committed secrets (P2). None of them survives a Git host today, though. The Git bridge refuses private branches, drops secrets, and exports protected files as bare ciphertext that loses its rules. sc-forge only stores plain Git.
This PR lets any Git repository carry a native sc store. Run
sc remote add origin sc+git::<git-url>, orsc clone sc+git::<git-url> <dst>. Push, fetch and clone then move private branches, protected paths and secrets through that repository without loss.How it works:
refs/sc/<key>. A plaingit clonenever fetchesrefs/sc/*. Publicmaincan still sit in the same repository as an ordinary Git branch.git push --force-with-lease, and never with a+refspec: a+makes git ignore a wrong lease. Two racing CAS writers get exactly one winner, and receive-pack enforces this under its ref lock. A rejected push is classified by re-reading the ref. A hook refusal is therefore an error, never a lost race.--allow-private-branchesonremote addorclone, orsc remote set. The branch name, recipient ids and sizes become readable by everyone who can read the Git repository, and the ciphertext stays there after a revoke. ADR-0060 records why this reverses ADR-0044's reason (4) only under that opt-in.refs/sc/prefix.sc remote compactandsc serve --storerefusesc+git::remotes. Compaction needs an age source the compactor controls. A daemon cannot answer ssh or credential prompts.The system
gitbinary is called fromobjio, and gix stays quarantined ingitio. Hardening covers the transport allowlist, environment scrubbing, value caps and URL redaction. ADR-0060 and THREAT-MODEL list the limits: everyone with push access can rewrite or delete the store, deleting a ref erases nothing, SHA-1 repositories only, and the whole ref list is downloaded on each call. The "not independently audited, no production secrets" warning stays.Proof:
demo/run_git_carrier_demo.shpasses twice against a bare repo and a real sc-forge over smart HTTP. A keyless clone sees only a branch marker and ciphertext, a keyed clone reads all three features, and the run leaves zero residue. New tests cover CAS races, hook refusal, decoy refs, corrupt values, URL and key validation and git-call counts. The fullcargo test --workspace, clippy-D warningsand fmt pass. The plan was reviewed by Fable, Mythos (security), Astra and code-reviewer. The diff was reviewed by Mythos and code-reviewer and with sol-loop.Not tested: real GitHub. Its acceptance of custom
refs/sc/*, its per-push limits and its CAS atomicity across replicas are unverified and documented as a follow-on.Reviewed with sol-loop: 2 rounds with gpt-6.2-sol on high. Sol verdict: approved
Created with Claude Opus 5.5 in Claude Code (T3 Code), with Claude Sonnet 5.5 subagents writing most of the implementation.