google_genai: sync generate_content likely emits one span per automatic-continuation hop (2.34+) #1041

Summary

google-genai 2.34.0 enables automatic continuation by default for generate_content. If the model stops with finish_reason == CONTINUATION and a continuation_token, the SDK sends follow-up requests and merges the responses. Because of where we patch, sync and async calls are traced differently:

• Sync (client.models.generate_content): we patch the private, per-request Models._generate_content, so we probably emit one span per continuation hop, each with partial output and usage.
• Async (client.aio.models.generate_content): we patch the public AsyncModels.generate_content, so we emit one span with the merged response.

Found while reviewing the weekly dependency bump in #1037 (google-genai 2.33.0 → 2.34.0). This hasn't been reproduced yet; step 1 below is a repro.

Upstream code (google-genai 2.34.0, google/genai/models.py)

def _generate_content_with_continuation(self, *, model, contents, config=None):
    enable_continuation = _extra_utils.should_enable_automatic_continuation(config, default_enabled=True)
    continuation_token = None
    responses = []
    while not responses or (enable_continuation and continuation_token):
        call_config = _extra_utils.prepare_continuation_config(config, continuation_token)
        response = self._generate_content(model=model, contents=contents, config=call_config)  # <- our patch point
        responses.append(response)
        continuation_token = _extra_utils.should_continue_generation(response)
    return _extra_utils.merge_continuation_responses(responses, config=config)

The public Models.generate_content calls this, either directly or once per automatic-function-calling (AFC) turn. Users can opt out with GenerateContentConfig(automatic_continuation=False).

Our code

• Sync patch target: ModelsGenerateContentPatcher uses target_path = "Models._generate_content".
• Async patch target: AsyncModelsGenerateContentPatcher uses target_path = "AsyncModels.generate_content".
• Wrappers: _generate_content_wrapper and _async_generate_content_wrapper.
• Streaming is probably fine: upstream's StreamContinuation makes usage_metadata cumulative across hops, which matches our last-chunk-wins aggregation in tracing.py. Worth asserting in the same test.

The sync side has the same structure for AFC: each AFC model turn is a separate _generate_content call. Check whether the per-request sync patch is intentional (it may exist to get one span per model turn under AFC). If it is, the fix may be to group continuation hops under one span rather than to move the patch point.

Suggested approach

1. Repro: write a cassette-backed test that triggers continuation. One option is a long-output prompt with a small max_output_tokens, if that produces CONTINUATION; check should_continue_generation for the exact trigger. Run it sync and async and compare span counts and outputs.
2. Decide the target shape: one span per user-level generate_content call with merged output and summed usage, or a parent span with a child per hop. Make sync and async match.
3. Implement. Options include:
   • patch Models._generate_content_with_continuation (sync) and AsyncModels._generate_content_with_continuation (async);
   • keep the per-request patch but suppress nested spans while a continuation loop is active.
     Gate on the method existing: older matrix pins (1.30.0, 1.75.0) don't have it.
4. Run cd py && nox -s "test_google_genai(latest)" and the older pins. Note that latest now requires Python 3.11+ (skip added in chore(deps): weekly dependency update #1037).