Repository navigation
perf: cut an sc+git:: push from 19 git round trips to 10 - #373
Merged
Merged
Conversation
… read-back after a push (sc+git::) A push to a sc+git:: carrier paid one git round trip per key. This cuts the network calls of a first push from 19 to 10 and of a second from 11 to 8. - Bucket gains put_new_batch and exists_batch (defaults loop put_new and exists; both in the contract suite). GitBucket overrides them: one `git push --porcelain` per chunk (64 MiB of values, the leased delete's item and command-line bounds), a create-only lease per ref, never `+`, never `--atomic`; the per-ref porcelain lines are read whatever git's exit status. Keys not reported landed are classified together by one ls-remote of their exact refs; if that fails the push refusal is returned, never false. exists_batch is one ls-remote per chunk, exact refname compare, never answered from the local state. - BucketTransport uploads pack and idx with one put_new_batch and checks every pending pack and idx with one exists_batch before the manifest CAS. - After a won manifest CAS the view is built from the manifest just written (build_view, split out of refresh_once) instead of reading it back; a build failure falls back to a plain refresh. - has_object (and only it) answers "absent" for 2 s after a manifest read that found the remote empty, instead of probing once per object.
Tighten the batch tests after breaking each rule on purpose (the per-ref porcelain lines read on a non-zero exit, check_key before any git call, what landed being cached), and record the measured counts in ROADMAP and ADR-0060. ROADMAP also declines the decoy-branch push failure and notes the old-pack download in ref_guard::check_ref_move as out of scope. ADR-0046 notes the 2 s empty-remote memo of has_object.
- A batch push names every rejected ref, and the first refusal of a batch is the one reported. - After a won CAS with an empty tag (an S3-compatible host that sent no ETag), re-read instead of building a view on it. - The memo tests no longer depend on setup finishing inside the 2 s window. - ADR-0060 says which backends pay the per-object manifest probe.
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars.
|
Codex Review Summary: ✅ CompletedThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
tonibergholm
approved these changes
Nov 22, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A push to an
sc+git::remote was slow on GitHub: a first small push took about a minute, at one HTTPS round trip per git call. Counted with a loggingSC_GITagainst a local bare repo, a first push into an empty carrier made 19 network git calls and a second push made 11. Ten of the first push's calls were five identical "is there a manifest?" probes, each a fetch plus anls-remote.This PR cuts that to 10 and 8, without loosening any CAS or compaction check:
Bucket::put_new_batch(the default loopsput_new).GitBucketsends one porcelain push per bounded chunk: at most 64 MiB of values, with the same item and command-line bounds asdelete_leased. Each ref gets its own create-only--force-with-lease, never a+refspec or--atomic. A ref not reported landed is classified by one exact-namels-remote, as a singleputis today.Bucket::exists_batchchecks every pending pack and idx in onels-remote. It never answers from the cache.has_objecttrusts a recent "empty". It answers from a 2 s memory of an empty remote.list_refs,head_branch,update_ref,get_packand compaction still read the manifest every time.A batch that fails can leave any subset of its keys created. Those are orphans that
sc remote compactsweeps, as a failed pair of single pushes leaves today.ROADMAP records (11) as built. It also declines the decoy-branch issue: a
refs/heads/refs/sc/Kshadowsrefs/sc/Kin git's push destination matching, and git cannot turn that off. Only someone who can already deleterefs/sc/*can plant one, and reads compare exact ref names. The old-pack download incheck_ref_moveis noted as a separate bandwidth item.Proof:
mainand this branch, and a clone afterwards reads both commits.has_object, and the view after a CAS.-D warnings,cargo test --workspace(1954 passed) anddemo/run_git_carrier_demo.shpass.The plan was reviewed by Mythos (security), Astra and code-reviewer, and the diff by code-reviewer.
Reviewed with sol-loop: 2 rounds with gpt-6.2-sol on high. Sol verdict: approved
Created with Claude Opus 5.5 in Claude Code (T3 Code), with a Claude Sonnet 5.5 subagent writing most of the implementation.