Skip to content

feat: carry private branches, protected files and secrets through any Git host (sc+git::) - #369

Merged
tonibergholm-codento merged 15 commits into
mainfrom
feat/sc-git-carrier
Nov 21, 2026
Merged

tonibergholm-codento merged 15 commits into
mainfrom
feat/sc-git-carrier

Conversation

@tonibergholm-codento

Copy link
Copy Markdown
Collaborator

Theo asked for three things in a Git host: files only some teammates can read, branches only some teammates can see, and built-in secrets. sc already has all three: protected paths (P7), private branches (P34) and committed secrets (P2). None of them survives a Git host today, though. The Git bridge refuses private branches, drops secrets, and exports protected files as bare ciphertext that loses its rules. sc-forge only stores plain Git.

This PR lets any Git repository carry a native sc store. Run sc remote add origin sc+git::<git-url>, or sc clone sc+git::<git-url> <dst>. Push, fetch and clone then move private branches, protected paths and secrets through that repository without loss.

How it works:

  • Storage. The existing bucket WAL remote (ADR-0046) already carries all three features. It is stored in the Git repository with one parentless commit per bucket key, at refs/sc/<key>. A plain git clone never fetches refs/sc/*. Public main can still sit in the same repository as an ordinary Git branch.
  • Atomic writes. Writes go through git push --force-with-lease, and never with a + refspec: a + makes git ignore a wrong lease. Two racing CAS writers get exactly one winner, and receive-pack enforces this under its ref lock. A rejected push is classified by re-reading the ref. A hook refusal is therefore an error, never a lost race.
  • Private-branch opt-in. Pushing a private branch needs a per-remote opt-in: --allow-private-branches on remote add or clone, or sc remote set. The branch name, recipient ids and sizes become readable by everyone who can read the Git repository, and the ciphertext stays there after a revoke. ADR-0060 records why this reverses ADR-0044's reason (4) only under that opt-in.
  • sc-forge. sc-forge now accepts the refs/sc/ prefix.
  • Not in v1. sc remote compact and sc serve --store refuse sc+git:: remotes. Compaction needs an age source the compactor controls. A daemon cannot answer ssh or credential prompts.

The system git binary is called from objio, and gix stays quarantined in gitio. Hardening covers the transport allowlist, environment scrubbing, value caps and URL redaction. ADR-0060 and THREAT-MODEL list the limits: everyone with push access can rewrite or delete the store, deleting a ref erases nothing, SHA-1 repositories only, and the whole ref list is downloaded on each call. The "not independently audited, no production secrets" warning stays.

Proof: demo/run_git_carrier_demo.sh passes twice against a bare repo and a real sc-forge over smart HTTP. A keyless clone sees only a branch marker and ciphertext, a keyed clone reads all three features, and the run leaves zero residue. New tests cover CAS races, hook refusal, decoy refs, corrupt values, URL and key validation and git-call counts. The full cargo test --workspace, clippy -D warnings and fmt pass. The plan was reviewed by Fable, Mythos (security), Astra and code-reviewer. The diff was reviewed by Mythos and code-reviewer and with sol-loop.

Not tested: real GitHub. Its acceptance of custom refs/sc/*, its per-push limits and its CAS atomicity across replicas are unverified and documented as a follow-on.

Reviewed with sol-loop: 2 rounds with gpt-6.2-sol on high. Sol verdict: approved

Created with Claude Opus 5.5 in Claude Code (T3 Code), with Claude Sonnet 5.5 subagents writing most of the implementation.

The pre-receive policy now accepts create, update (including
non-fast-forward) and delete for names under exactly `refs/sc/`, so a
repository hosted on sc-forge can carry a native sc bucket store.
`refs/sc`, `refs/scx/x` and every other namespace stay refused; heads and
tags are unchanged.

End-to-end tests drive the real receive path over smart HTTP with plain
git: create with an empty lease, CAS update without `+`, stale lease
rejected, exact-refspec fetch, delete, two racing CAS pushes with one
winner, a hand-built receive-pack request that proves the server checks
the old value under the ref lock, and a maintenance pass that keeps the
live values and prunes the replaced one.

ADR-0058 addendum, FORGE-OPERATIONS.md and THREAT-MODEL.md describe the
namespace: writable by every repository writer, outside branch
protection, advertised to readers, bounded by receive.maxInputSize.
…itory

Adds the objio backend for the sc+git:: carrier (P39). Each bucket key lives
at refs/sc/<key> as a parentless commit whose tree holds the value as
chunk blobs (32 MiB each, 2 GiB per value). Create-only writes push with an
empty lease and CAS writes with the expected oid as the lease, never with a
+ refspec; every rejection is classified by re-reading the exact ref with
ls-remote, so a hook refusal is an error and never a lost race.

The backend drives the system git binary (SC_GIT overrides), with a scrubbed
GIT_* environment, a protocol allowlist, no hooks or signing, a fixed carrier
identity and a private temp cache repository removed on drop. Cold reads of
log/, checkpoints/ and packs/*.idx fetch the whole family in one call.

Tests cover the contract suite, racing CAS and create, stale and malformed
tags, a rejecting pre-receive hook, decoy refs, chunking, corrupt value
shapes, URL and key validation, hostile user git config, cache cleanup and a
counting git shim that bounds network calls independent of entry count.
… history

A glob fetch of a key family now uses --depth=1 like exact fetches. A shallow
glob that matches nothing exits 1, so a failed family fetch lists the family
once: empty means absent, anything else is a real failure.
…nch opt-in

Route `sc+git::<git-url>` to the bucket transport (GitBucket stores the WAL
as refs/sc/* in any Git repository). `sc+git:` is a reserved prefix: only
the exact `sc+git::<inner>` form parses, and a near miss is refused instead
of being saved or read as a local path or sent to the P18 Git bridge.

- one `is_bucket_url` replaces the duplicated scheme checks
- `remote add --git` and `clone --git` refuse sc+git::
- per-remote `allow_private_branches` in .sc/config (absent = false, omitted
  on write while false); `remote add`/`clone --allow-private-branches` set it
- `Repo::push` refuses a private branch on a gated remote before any I/O,
  naming the flag and what becomes readable
- `remote add`/`clone` print a one-line exposure notice
- `remote compact` and `serve --store` refuse sc+git:: (deferred)
- bucket errors are escaped for the terminal and no longer expose the raw
  inner error as an anyhow source
- end-to-end tests over a bare Git repo are #[ignore]d until the real
  GitBucket lands
ADR-0060 records the refs/sc/* mapping, lease CAS, private-branch opt-in,
deferred compaction and serve --store, and the dependency-rule amendment.
CLAUDE.md, ARCHITECTURE, THREAT-MODEL, ROADMAP, README and the sc-cli-reference
skill carry the capability row, boundaries and follow-ons.
Self-checking proof for P39: a bare Git repo and a real sc-forge carry a
protected path, a secret and a private branch as refs/sc/*. Covers the
private-branch opt-in gate, a plain git clone that sees no sc store, keyless
and keyed sc clones, the Git bridge mirroring public main in the same repo,
and zero sc-git-bucket-* residue across two runs.
- Drop a key's cached tag after any write that does not land, so the 2 s
  freshness window cannot answer Unchanged for a tag that just lost a CAS.
- Decide create-only by key layout (log/, checkpoints/, packs/*.pack|idx),
  never by the writer-controlled commit message.
- Treat a refused push as success when the remote already holds our new
  commit (lost acknowledgement); classification is a pure function.
- Require a single-key ref to point directly at a commit; no tag peeling.
- Force protocol.ext/fd/git.allow=never and fetch.fsckObjects=true.
- URL validation: no userinfo on http(s), no leading '-' in ssh/scp user,
  restricted ssh:// host charset.
- Cap every non-.pack value at 256 MiB, checked before reading.
- Check family ref names before they enter state; match GIT_* env names
  case-insensitively on Windows.
- Export redact; note SIGINT in the module docs.
- Tests: counting shim now covers the writer too (the zero-call assertion
  was vacuous).
… remote set

- Never print a raw sc+git URL: parse errors, the private-branch gate, the
  --git bails and "added remote" show it with userinfo redacted.
- Push reads the branch and tip once, gates on that tip and pushes it.
- Compaction is refused in BucketTransport itself for sc+git:: remotes, not
  only in the CLI.
- The reserved sc+git: prefix is case-insensitive; only exact lowercase
  sc+git::<inner> parses.
- Add `sc remote set <name> --allow-private-branches` and
  `--no-allow-private-branches`; the gate message names it.
- CLI tests assert the partial-clone refusal text and that the e2e run
  leaves no sc-git-bucket-* behind in its TMPDIR.
ADR-0060, ARCHITECTURE, THREAT-MODEL, ROADMAP and the sc-cli-reference
skill: lost-ack and layout-only create-only rules, userinfo refusal on
http(s), forced protocol denials, 256 MiB cap for non-pack values, and
`sc remote set`. Record as a v1 limit that every ls-remote and push
downloads the whole ref advertisement (the host does not filter by prefix)
and a cold read fetches every log and checkpoint entry, with refs growing
by at least three per push while compaction is refused.
…ed value

A user default of init.defaultObjectFormat=sha256 made the cache sha256, so
every push to a sha1 carrier failed. Confirming a newer tag also refreshed the
cache entry of an older value, which could then be answered as unchanged.
SHA-256 carriers are a documented v1 limit.
@coderabbitai

coderabbitai Bot commented Nov 21, 2026 •

Copy link
Copy Markdown

Important

  • Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: bcde453a-1172-4ed5-b4c4-fb65dca5d756
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 21, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-21T10:54:57.181881Z a89836f New commits
🔒 Security Review ✅ Completed 2026-11-21T11:01:30.929222Z a89836f New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2bd01df667

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread crates/objio/src/git.rs Resolved
Comment thread crates/forge/tests/e2e_git.rs Fixed
Comment thread crates/forge/tests/e2e_git.rs Fixed
Comment thread crates/forge/tests/e2e_git.rs Fixed
Comment thread crates/forge/tests/e2e_git.rs Fixed
Comment thread crates/forge/tests/e2e_git.rs Fixed
Comment thread crates/forge/tests/e2e_git.rs Fixed
Comment thread crates/forge/tests/e2e_git.rs Fixed
Comment thread crates/forge/tests/e2e_git.rs Fixed
Comment thread crates/forge/tests/e2e_git.rs Fixed
Comment thread crates/forge/tests/e2e_git.rs Fixed
CodeQL flags a literal used as a nonce. The forge refs/sc tests now draw it
from scl_crypto::random_hex, as the repo does for test passwords.

@tonibergholm tonibergholm left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved after agent reviews (Fable, Mythos, Astra, code-reviewer, sol-loop) and green CI on a89836f.

@tonibergholm-codento
tonibergholm-codento merged commit ab23a6a into main Nov 21, 2026
11 checks passed
@tonibergholm-codento
tonibergholm-codento deleted the feat/sc-git-carrier branch November 21, 2026 11:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants