Skip to content

feat(builder): declare uv.lock git sources for the caller to mirror - #103

Merged
saurabhmaheshwari-obin merged 11 commits into
codefly-dev:mainfrom
saurabhmaheshwari-obin:feat/declare-uv-git-sources
Oct 23, 2026
Merged

saurabhmaheshwari-obin merged 11 commits into
codefly-dev:mainfrom
saurabhmaheshwari-obin:feat/declare-uv-git-sources

Conversation

@saurabhmaheshwari-obin

@saurabhmaheshwari-obin saurabhmaheshwari-obin commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

Depends on codefly-dev/core#832 (the GitSourceDownload declaration, contract v6), released in core v0.19.6, which go.mod pins. Core requires Go 1.27.3, so the go line moves to it. ci.yml's go-service-ci.yml and the AGENTS.md note move to @v0.19.6 with it.

Release order: release the CLI change (codefly-dev/cli#1106) before this agent. An older CLI refuses a v6 plan, so a service whose uv.lock has a git source would stop building against it.

What

The builder declares the git sources pinned in uv.lock, so the caller can fetch them on the host with its own credentials and hand the build a mirror of each. The image build then resolves them with no credential.

  • Declare (uvgitsources.go, builder.go):
    • read uv.lock (the workspace root's lock for a workspace member);
    • for each [[package]] source.git: repository = the URL exactly as uv records it up to ? or #; commit = the fragment;
    • de-duplicate and sort;
    • declare WithGitSourceDownloads(..., MirrorContext: "uvsources") on both build paths.
  • Template (Dockerfile.tmpl), only when sources are declared:
    • the pinned # syntax= line (the same digest the go-grpc template pins) and a FROM scratch AS uvsources stage;
    • the uv sync step mounts it and, when the mirror is non-empty, appends one url.file:///uvsources/<GitSourceMirrorPath>.insteadOf=<repository> entry per repository to GIT_CONFIG_*, plus GIT_TERMINAL_PROMPT=0, then runs the unchanged uv sync.
    • A service without git sources renders a byte-identical Dockerfile (golden).

Why

Issue #102. The builder stage runs uv sync --frozen with no credential, so a private git source cannot be built today. The caller already has the credential. Companion PRs:

The URL form was determined with uv 0.12.16, this agent's pin, in the builder image, by logging the arguments uv hands to git:

  • uv.lock records https://host/org/repo?subdirectory=…&rev=<sha>#<sha>, and uv fetches https://host/org/repo with +<sha>:refs/commit/<sha>;
  • .git and a trailing slash are kept;
  • the host is lowercased;
  • git+ssh is recorded as ssh;
  • scp-style URLs are refused by uv itself.

The mirror key is that exact URL.

Deviations from the Go module download path

This change follows the recipe-declared Go module download:

  • the same Build-time declaration;
  • the same named build context, fetched by the caller with its own credentials;
  • no credential inside the build;
  • contract versioning only when declared (v6 here, v5 for Go);
  • an empty FROM scratch stage that falls back to the network for older callers and a plain docker build.

Where it differs:

  • Declared for workspace members too. Go declares nothing for a workspace build, because its module root is outside the build context. Here the assembled context carries the workspace root's uv.lock, which uv sync reads in the build, so the root lock's git sources are declared. That can be more than one member needs.
  • Stage and frontend pin only when declared. Go always emits gomodproxy under a pinned # syntax=. Here the stage, its mount and the same pinned frontend appear only when the lock has a mirrorable git source, so every other service renders a byte-identical Dockerfile.
  • Per-repository insteadOf, not a proxy URL. git and uv have no module-proxy protocol. Each declared repository maps onto its bare mirror through GIT_CONFIG_*, using the URL exactly as uv.lock records it. As with GOPROXY=file://, a declared commit missing from the mirror fails the build rather than reaching the network.
  • Mappings are appended to inherited git configuration. Go overwrites the proxy settings outright. Git configuration is additive, so the mappings are added after any GIT_CONFIG_* entries the image carries, and those entries are left untouched. The inherited count must be a plain number (a leading + and leading zeros allowed, at most four digits). Otherwise the configuration is not touched, the build log says so, and the build fetches as it does today.
  • Sources core would refuse are not declared. The agent applies core's own checks (GitSourceRepositoryProblem, IsGitSourceCommit) before declaring. A source that fails them (http, file, a credential in the URL, percent-encoding, @ in the path) is named with core's reason in a warning, never by its URL, and fetched by the build as before. A commit that is not a full object id is a malformed lock and fails the build. Go has no equivalent case.
  • Prefix-capturing sources are not declared. git's insteadOf matches by prefix, so a URL that begins another locked source's undeclared URL is left out, with a warning, so it cannot redirect that source into the wrong mirror.
  • No fallback credential. This builder never mounted one, so there is no GOPRIVATE or netrc equivalent to keep.

Known limitation: an image whose own git configuration (system, global or environment) already rewrites a declared repository's URL keeps its rewrite: git prefers it, and the build fetches that source as it does without a mirror. That is today's behaviour, not a regression.

Testing

  • go build, go vet and gofmt are clean; go mod tidy -diff is clean.
  • go test ./...: 139 pass. The only skip is the SBOM test, which needs syft. go test -race ./... passes.
  • Unit tests cover:
    • the lock's sources declared once and sorted (subdirectory packages share a repository and commit; the same repository at two commits; ssh with a port and .git; a registry package);
    • refused sources unmirrored with a warning, and absent from the plan and the Dockerfile (a token in the URL, %20, @v1, http, file);
    • prefix capture, including chains and the longest declared match;
    • a missing or short commit fails, naming the package;
    • the workspace member reads the root lock;
    • goldens with and without sources.
  • Docker tests, run in the real builder image:
    • TestLockedGitSourceBuildsFromTheSuppliedMirror. A real uv.lock, made by the host's uv through an insteadOf rewrite, pins a git source at a URL the build cannot reach. With --build-context uvsources=<mirror> the build succeeds, and the image reports the package; without it, the build fails ("Could not resolve host").
    • TestGitSourceMappingInTheBuilderImage. Inherited entries survive, and counts 1, 01 and +1 append. For 1, -0, 99999 and abc, the configuration is untouched and the build log says so. The precedence limitation is pinned. Prefix capture does not rewrite undeclared sources.
  • The existing TestAssembledContextBuildsTheImage, TestWorkspaceMemberBuildsTheImage and TestCreateToRunDocker pass.
  • Size: about 245 hand-written lines; the rest is tests (about 390) and generated goldens (about 150).

Not verified:

  • git submodules of a declared repository. A submodule URL that a declared URL prefixes would be rewritten, and private submodules are not mirrored. The lock does not list submodules, so the prefix check cannot see them.
  • LFS-backed sources.
  • Credentials written into pyproject.toml. If uv keeps them in the lock, core's check refuses the source, and it is left to the network.

Review

  • Side-by-side check against the Go module download path (go-grpc's declaration, template and tests): aligned. The deviations above are the complete list, each with its reason.
  • Codex (xhigh), four rounds:
    • a token rendered before validation, and valid uv URLs that core refuses: fixed by pre-validating with core's exported checks;
    • inherited git configuration overwritten, and prefix capture: fixed;
    • precedence against an existing rewrite: simplified to append-only and documented as above;
    • count parsing: bounded and validated;
    • final round: no P1, P2 or P3. The rendered snippet passed 37 cases each under BusyBox sh, dash and bash (hostile strings, leading zeros, huge counts, missing entries, empty mirrors); no-source Dockerfiles are byte-identical.

🤖 Generated with Claude Code

Pins github.com/codefly-dev/core to the open change that adds
DockerBuildRecipe.git_source_downloads (contract v6), by pseudo-version,
until it is released; the pin moves to the core tag, together with the
shared workflow in ci.yml, after the release.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A git dependency that needs a credential resolves on the host and fails
in the builder stage, which has none. The recipe now declares every git
commit uv.lock pins (git_source_downloads, mirror context "uvsources"):
the caller fetches them on its own machine before the build and supplies
one bare mirror per repository, and the builder stage maps each locked
URL onto its mirror with url.<mirror>.insteadOf before the unchanged
uv sync --frozen.

The declared repository is the lock's URL up to its query, which is the
string uv hands git, so the insteadOf match is exact. The workspace
root's lock is read for a workspace member. A lock with no git sources
renders the Dockerfile and plan contract it rendered before, and a plain
docker build without the context fetches from the network as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Only https and ssh git sources can be mirrored. A package locked to any
other git URL was refused, which would have broken a project that builds
today. It is now left out of the declaration, with a warning naming the
package (not the URL), and the build fetches it from the network as
before. A locked git source without a commit is still refused: uv always
records one, so that lock is malformed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A locked source core refuses (a credential over https, percent-encoding,
"@" in the path, a scheme other than https or ssh) was declared and
rendered into the Dockerfile before core refused the plan, failing a
build that works today. Each source is now checked with core's own
GitSourceRepositoryProblem first: a refused one is neither declared nor
rendered, and a warning names the package and core's reason, never the
URL; the build fetches it itself, as before. A commit that is not a full
object id (IsGitSourceCommit) still fails, naming the package.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The sync step set GIT_CONFIG_COUNT and numbered its insteadOf mappings
from 0, replacing any git configuration a base image or an earlier ENV
had put in the environment. The mappings are now appended after
${GIT_CONFIG_COUNT:-0}, and the count is raised to cover them. Values
stay single-quoted.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ared

The sync step's RUN --mount comes from the Dockerfile frontend, so a
recipe that declares git sources pins it, with the same digest the Go
agent's recipe pins. A recipe without git sources renders as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…her git config

git's insteadOf rewrites every URL that begins the configured one, so a
declared https://h/acme/mono also rewrote an undeclared locked
https://h/acme/mono@v1 onto its mirror, failing a build that worked.
A declarable source whose URL begins an undeclared locked source's URL
is now left out as well (warned, fetched by the build), until no such
capture remains. Two declared URLs, one beginning the other, stay
declared: git applies the longest match.

An inherited rewrite of the same URL came first and won over the
appended mirror mapping. The mappings now take entries 0..N-1 and the
inherited entries move after them, read by constructed name and never
evaluated. The inherited count is read as git reads it (a leading "+"
and leading zeros, so 01, +1 and 08 work); a count git would reject is
left alone for git to reject.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… count

Moving inherited git config after the mirror mappings could not win
precedence anyway (git breaks insteadOf ties by the order the
destination base was registered, so a system or global rewrite still
wins) and added edge cases of its own. The mappings are appended after
the inherited entries again, without reading or moving them. They are
used only when GIT_CONFIG_COUNT is unset, empty, or a plain count of at
most four digits once one leading "+" and leading zeros are dropped;
otherwise the git configuration is left alone, the build log says the
sources are fetched without the mirror, and the build runs as it does
today.

An image whose own git configuration (system, global or environment)
already rewrites a declared repository's URL keeps its rewrite: git
prefers it, and the build fetches that source as it does without a
mirror.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ation (#832)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@saurabhmaheshwari-obin
saurabhmaheshwari-obin merged commit 3ba30e8 into codefly-dev:main Oct 23, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant