Repository navigation
Conversation
java-storage ignores GOOGLE_CLOUD_UNIVERSE_DOMAIN and the credentials' universe domain, so GcsUtilV2 always targeted storage.googleapis.com. When no gcsEndpoint is set, resolve the universe domain (env var, then credentials) and set it on StorageOptions. Add unit tests.
There was a problem hiding this comment.
📋 Review Summary
This PR resolves an issue in GcsUtilV2 where non-default universe domains (such as Trusted Partner Cloud) were ignored when --gcsEndpoint was not explicitly configured. It properly resolves the universe domain from GOOGLE_CLOUD_UNIVERSE_DOMAIN or the configured Credentials and applies it to StorageOptions, while preserving --gcsEndpoint precedence and default googleapis.com behavior.
🔍 General Feedback
- The resolution precedence (
--gcsEndpoint>GOOGLE_CLOUD_UNIVERSE_DOMAIN>Credentials#getUniverseDomain()> default) is clean and consistent with the Google API client library behavior and the Python SDK implementation (#40674). - Unit tests in
GcsUtilV2Testthoroughly cover environment variable precedence, credential fallback, default universe handling, error handling ongetUniverseDomain(), and--gcsEndpointoverride.
💡 Tip: Comment /gemini review on this pull request to trigger a new Gemini code review.
|
Assigning reviewers: R: @damccorm for label java. This pull request likely touches a core component ("core" label). Please review with scrutiny. Note: If you would like to opt out of this review, comment Available commands:
The PR bot will only process comments in the main thread (not review comments). |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #40796 +/- ##
===========================================
- Coverage 80.04% 0 -80.05%
===========================================
Files 490 0 -490
Lines 82916 0 -82916
===========================================
- Hits 66373 0 -66373
+ Misses 16543 0 -16543
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Part of #40569
Problem
Since GcsUtil V2 (java-storage) became the default (#40713), Java GCS ignores
GOOGLE_CLOUD_UNIVERSE_DOMAIN. Unlike the Google API client used by GcsUtil V1 (and by the Dataflow client, #40672), java-storage only takes the universe domain fromStorageOptions#setUniverseDomain. Without--gcsEndpoint, every request goes tostorage.googleapis.com, even with credentials for another universe.On Trusted Partner Cloud this fails right at pipeline construction:
Fix
When
--gcsEndpointis not set,GcsUtilV2resolves the universe domain and sets it onStorageOptions:GOOGLE_CLOUD_UNIVERSE_DOMAIN, if set; otherwiseIf neither is set, or the value is
googleapis.com, nothing changes. An explicit--gcsEndpointstill takes precedence. This mirrors the Python fix ([Python] Fix GCS client creation in non-default universes #40674).