Skip to content

feat(gapic-generator): add support for resumable uploads - #18354

Draft
parthea wants to merge 25 commits into
mainfrom
feat/resumable-transfer-gapic-generator
Draft

parthea wants to merge 25 commits into
mainfrom
feat/resumable-transfer-gapic-generator

Conversation

@parthea

@parthea parthea commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

Towards b/457416314, b/556259599

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces support for resumable upload methods in the GAPIC generator, updating schema wrappers, templates, and options to handle media upload protocols, while adding comprehensive system and unit tests. Feedback on the changes suggests avoiding potential breaking changes by generating media upload methods as standard RPCs when resumable_upload_prefix is not configured, rather than omitting them entirely. Additionally, it is recommended to deduplicate helper functions like resume_resumable_upload and make_resumable_upload across the newly added system test files by moving them to a shared utility module or conftest.py.

Comment thread packages/gapic-generator/gapic/schema/api.py Outdated
Comment thread packages/gapic-generator/tests/system/test_resumable_upload_basic.py Outdated
@parthea parthea changed the title DRAFT feat(gapic-generator): add support for resumable uploads feat(gapic-generator): add support for resumable uploads Sep 14, 2026
dependencies = [
"google-api-core[grpc] >= 2.28.0, <3.0.0",
# TODO: For prototyping purposes only, revert once https://github.com/googleapis/google-cloud-python/pull/18352 is merged
"google-api-core[grpc]@git+https://github.com/googleapis/google-cloud-python.git@feat/resumable-transfer-api-core#subdirectory=packages/google-api-core",

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For prototyping purposes only, revert once #18352 is merged and released to PyPI

# and https://github.com/actions/checkout#checkout-head.
with:
fetch-depth: 2
fetch-depth: 0

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will revert this change once #18352 is merged to main and released

@daniel-sanche daniel-sanche left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible to break some of this up into separate PRs? There's a lot here

# - It may require correct/in-range values for request initialization.
# - It may require specifying regional endpoints when creating the service
# client as shown in:
# https://googleapis.dev/python/google-api-core/latest/client_options.html

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it standard to include this text within the region tags of every sample? Is this because it's in showcase?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it is standard. It exists on all generated samples. See details in go/snippet-gen-design


# Initialize request argument(s)
request = showcase_v1beta1.EnumRequest(
)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we need to run ruff on these? The formatting seems unexpected

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't run ruff on golden files. It's a post processing step (post generation). There is a similar formatting issue in other goldens. I will file a bug to follow up on whether we want to run ruff on goldens but let's tackle that separately since it's an existing issue.


# Iterate over the upload to receive progress updates as each chunk is transmitted
stream = io.BytesIO(b"Example upload data")
for progress in upload_session.iter_upload(stream):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: it would be helpful to add a type annotation here

retry: OptionalRetry = gapic_v1.method.DEFAULT,
timeout: Union[float, object] = gapic_v1.method.DEFAULT,
metadata: Sequence[Tuple[str, Union[str, bytes]]] = (),
) -> resumable_upload.UploadMediaResponse:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems to have a different signature than tha samples (i.e. it doesn't accept config, doesn't return ResumableUploadSession)

self.get_operation,
default_timeout=None,
client_info=client_info,
kind='rest',

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't look right, this is the grpc_asyncio transport

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants