Skip to content

gocache/awscache: add a DynamoDB+S3 Go build cache library - #12

Merged
bradfitz merged 1 commit into
mainfrom
bradfitz/awscache
Nov 2, 2026
Merged

bradfitz merged 1 commit into
mainfrom
bradfitz/awscache

Conversation

@bradfitz

Copy link
Copy Markdown
Member

Add a library for a Go build cache that keeps action metadata in
DynamoDB and objects in S3, meant to be used directly by the process
serving a client's cache requests (in Tailscale's case, ciguestlet on
the CI hosts) rather than through a central cache server.

It has no auth or policy: callers pass a Grant naming the namespaces a
request may read and write. Handler serves gocached's HTTP protocol, so
existing cachers.HTTPClient users (cigocacher) work unchanged.

Storage is one DynamoDB item per (namespace, action), with objects up
to 700 bytes inline and larger ones as content-addressed, lz4-compressed,
write-once S3 blobs that readers verify by SHA-256 before serving. GET
hits bump access times and DynamoDB TTLs relatime-style in the
background, so unused actions expire; blob deletion is left to a
separate scrubber. Objects pass through memory, or disk past a memory
budget, but nothing is cached locally.

Updates tailscale/corp#48514

Add a library for a Go build cache that keeps action metadata in
DynamoDB and objects in S3, meant to be used directly by the process
serving a client's cache requests (in Tailscale's case, ciguestlet on
the CI hosts) rather than through a central cache server.

It has no auth or policy: callers pass a Grant naming the namespaces a
request may read and write. Handler serves gocached's HTTP protocol, so
existing cachers.HTTPClient users (cigocacher) work unchanged.

Storage is one DynamoDB item per (namespace, action), with objects up
to 700 bytes inline and larger ones as content-addressed, lz4-compressed,
write-once S3 blobs that readers verify by SHA-256 before serving. GET
hits bump access times and DynamoDB TTLs relatime-style in the
background, so unused actions expire; blob deletion is left to a
separate scrubber. Objects pass through memory, or disk past a memory
budget, but nothing is cached locally.

Updates tailscale/corp#48514
@bradfitz
bradfitz requested a review from a team October 22, 2026 15:15
@bradfitz

bradfitz commented Nov 2, 2026

Copy link
Copy Markdown
Member Author

Per chat with @tomhjp, we've both reviewed this enough and are mostly happy with it, so we're going to merge it for now so we can proceed with testing the other side (in the corp repo). But we'll probably come back to this code and tweak its buffering logic a bit.

@bradfitz
bradfitz merged commit c7f76f3 into main Nov 2, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant