Repository navigation
The .git/ directory takes 4 GB per build on Fedora Stable x86-64 buildbot #789
Description
After seeing issue gh-788, I checked the size of the .git/ directory on the Fedora Stable x86-64 buildbot worker. It takes 4.1 GB, but this size is reduced to 274 MB after running gc gc.
bash-5.3$ du -sh .git
4.1G .git
bash-5.3$ git gc
(...)
bash-5.3$ du -sh .git
274M .git
I see that the "ARM64 Windows" and "wasm32-wasi" buildbots use a custom Git configuration to use a shallow clone:
git_options=dict(
# Do a full shallow clone for every build
mode="full",
method="clobber",
shallow=True,
# Disable the default, if set
filters=None,
),
Should we use a shallow clone on all buildbot workers to reduce the .git/ size? Currently, we use:
# "git clean -fdx": remove all files which are not tracked
# by Git, ignoring the .gitignore rules (ex: remove also
# ".o" files).
mode="full",
method="fresh",
# Partial (blobless) clone: only fetch commit and tree
# objects, blobs are fetched on demand during checkout.
filters=["blob:none"],
cc @zware @stratakis
Activity
zware commentedon Sep 9, 2026
If
git gcis effective, maybe we could add a post-cleanstep for that?zware commentedon Sep 14, 2026
I've implemented the
git gcstep in #790 and have been watching disk usage drop significantly on myware-debian-x86worker, so hopefully it's helping other workers as well.vstinner commentedon Sep 14, 2026
Aha, interesting. Let me check some buildbot workers:
Oh wow, I'm honestly impressed. Previouslty, the disk usage was between 84 GB and 146 GB. So between 21 and 24 GB is WAY better!
I also checked a single build directory on Fedora Stable x64-64:
.git/takes only275 MB. That's way better that the 4.1 GB I saw previously! I close the issue.