You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[Task]: Add a GCS temp bucket cleaner to .test-infra/tools to prevent bucket leaks #40347
In PR #40279, we started randomizing bucket names for GcsUtil tests to prevent collisions between concurrent test runs. The temporary buckets are now created with randomized names, such as apache-beam-temp-bucket-<UUID>.
Problem
While this randomization successfully resolves naming conflicts, it introduces a risk of bucket leaks. If a test crashes, is interrupted, or fails to clean up properly after execution, these randomized buckets will be left behind in the project indefinitely, unnecessarily consuming resources.
Possible Solution
As discussed in this review comment, we should build a mechanism to automatically sweep up these orphaned buckets:
Create a cleanup tool: Add a GCS temp bucket cleaner script or utility in the /.test-infra/tools directory. The tool should be responsible for identifying and deleting stale apache-beam-temp-bucket-* buckets (e.g., buckets older than a certain number of hours).
Integrate with CI: Add this cleaner as a step in the existing cleanUp workflow so that it runs regularly to automatically eliminate any leaked buckets.
Issue Priority
Priority: 2 (default / most normal work should be filed as P2)
@shunping i have a patch ready #40386 . It adds a 24-hour cleaner for leaked apache-beam-temp-bucket-<UUID> buckets and wires it into the scheduled cleanup workflow. Let me know if this looks good. thanks!
Activity
goutamadwant commentedon Oct 1, 2026
.take-issue
goutamadwant commentedon Oct 2, 2026
@shunping i have a patch ready #40386 . It adds a 24-hour cleaner for leaked
apache-beam-temp-bucket-<UUID>buckets and wires it into the scheduled cleanup workflow. Let me know if this looks good. thanks!