Skip to content

protocol: sign recipient file URLs for cloud storage - #108

Draft
littlegrasscao wants to merge 1 commit into
server/split2/17-protocol-table-credentialsfrom
server/split2/18-protocol-query-table
Draft

littlegrasscao wants to merge 1 commit into
server/split2/17-protocol-table-credentialsfrom
server/split2/18-protocol-query-table

Conversation

@littlegrasscao

@littlegrasscao littlegrasscao commented Sep 29, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

  • add UrlSigners in core, which picks a signer by path scheme and caps the URL lifetime to the catalog credential expiry
  • sign S3 object URLs with AWS SigV4 presigning
  • sign Azure blobs with the catalog SAS token, or a blob SAS from an account key
  • sign GCS objects with V4 signatures, using the catalog-vended service account key file if present, otherwise Google application default credentials (no server setting)
  • signers are plain classes, not wired into OpenSharing until the query endpoint uses them
  • AWS credentials now require region: the local catalog's FAKE mode returns us-west-2, STATIC requires it, and ENV requires AWS_REGION or AWS_DEFAULT_REGION
  • add the S3, Azure Blob, and Google Cloud Storage SDK dependencies

Testing

  • mvn clean install: 171 tests passed
  • S3UrlSignerTest, AzureSasUrlSignerTest, GcsUrlSignerTest, UrlSignersTest: signing offline with placeholder credentials, unknown schemes, and lifetime capping
  • UrlSignerCloudIntegrationTest signs a real data file from the S3, Azure, and GCS test tables and reads its first byte; each case is skipped without credentials
  • LocalCatalogConnectorTest checks the FAKE-mode region

Stacked on #107.

@littlegrasscao
littlegrasscao force-pushed the server/split2/18-protocol-query-table branch 2 times, most recently from 0a73f7d to 27f2865 Compare September 29, 2026 18:37
@littlegrasscao
littlegrasscao force-pushed the server/split2/18-protocol-query-table branch from 48e5f02 to 168d12b Compare September 29, 2026 21:59
@littlegrasscao littlegrasscao changed the title Query Table snapshot API Sign recipient file URLs for cloud storage Sep 29, 2026
@littlegrasscao
littlegrasscao force-pushed the server/split2/18-protocol-query-table branch from 95b8b31 to dbf6206 Compare September 29, 2026 22:06
@littlegrasscao
littlegrasscao marked this pull request as ready for review September 29, 2026 23:35
@littlegrasscao
littlegrasscao force-pushed the server/split2/18-protocol-query-table branch from 42a853b to 8a41489 Compare October 1, 2026 17:24
@littlegrasscao
littlegrasscao force-pushed the server/split2/18-protocol-query-table branch from 8a41489 to 218ae45 Compare October 1, 2026 18:34
@littlegrasscao
littlegrasscao force-pushed the server/split2/18-protocol-query-table branch from 218ae45 to 9233fb9 Compare October 1, 2026 18:39
@littlegrasscao
littlegrasscao force-pushed the server/split2/18-protocol-query-table branch from 9233fb9 to 4e36e66 Compare October 1, 2026 18:44
@littlegrasscao
littlegrasscao marked this pull request as draft October 7, 2026 01:57
@littlegrasscao littlegrasscao changed the title Sign recipient file URLs for cloud storage protocol: sign recipient file URLs for cloud storage Oct 9, 2026
@littlegrasscao
littlegrasscao force-pushed the server/split2/18-protocol-query-table branch from 4e36e66 to 39841fa Compare October 14, 2026 01:43
@littlegrasscao
littlegrasscao force-pushed the server/split2/18-protocol-query-table branch from 39841fa to bca6420 Compare October 14, 2026 02:07
@littlegrasscao
littlegrasscao force-pushed the server/split2/18-protocol-query-table branch from bca6420 to 3dad5a4 Compare October 14, 2026 19:25
@littlegrasscao
littlegrasscao force-pushed the server/split2/18-protocol-query-table branch from 3dad5a4 to 85ff185 Compare October 15, 2026 16:47
@littlegrasscao
littlegrasscao force-pushed the server/split2/18-protocol-query-table branch from 85ff185 to 58daefb Compare October 15, 2026 18:14
@littlegrasscao
littlegrasscao force-pushed the server/split2/18-protocol-query-table branch from 58daefb to 1b7dd09 Compare October 15, 2026 18:29
@littlegrasscao
littlegrasscao force-pushed the server/split2/18-protocol-query-table branch from 1b7dd09 to 9fa5e98 Compare October 15, 2026 23:35
@littlegrasscao
littlegrasscao force-pushed the server/split2/18-protocol-query-table branch from 9fa5e98 to 03c514f Compare October 15, 2026 23:38
@littlegrasscao
littlegrasscao force-pushed the server/split2/18-protocol-query-table branch from 03c514f to 31beb6a Compare October 15, 2026 23:54
@littlegrasscao
littlegrasscao force-pushed the server/split2/18-protocol-query-table branch from 31beb6a to 576aef6 Compare October 16, 2026 00:18
@littlegrasscao
littlegrasscao force-pushed the server/split2/18-protocol-query-table branch from 576aef6 to 9089369 Compare October 16, 2026 00:41
@littlegrasscao
littlegrasscao force-pushed the server/split2/18-protocol-query-table branch from 9089369 to 743022d Compare October 19, 2026 17:13
@zhu-tom
zhu-tom added this pull request to stack #116 October 19, 2026 19:06
@littlegrasscao
littlegrasscao force-pushed the server/split2/18-protocol-query-table branch from 743022d to c414c96 Compare October 19, 2026 21:44
@littlegrasscao
littlegrasscao force-pushed the server/split2/18-protocol-query-table branch from c414c96 to 1567f19 Compare October 19, 2026 22:35
@littlegrasscao
littlegrasscao force-pushed the server/split2/18-protocol-query-table branch from 1567f19 to 111a6f7 Compare October 19, 2026 22:37
@littlegrasscao
littlegrasscao force-pushed the server/split2/18-protocol-query-table branch 2 times, most recently from 111a6f7 to 7e61f1a Compare October 20, 2026 00:31
@littlegrasscao
littlegrasscao force-pushed the server/split2/18-protocol-query-table branch from 7e61f1a to 26ed6ec Compare October 21, 2026 17:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant