feat(observability): [WIP] implement universal 4-path OpenTelemetry tracing

[chalmerlowe]

feat(observability): implement universal 4-path OpenTelemetry tracing

Problems Solved

Google Cloud Python client libraries support four communication paths: synchronous gRPC, asynchronous gRPC, synchronous REST (HTTP), and asynchronous REST (HTTP). Previously, distributed OpenTelemetry tracing was only wired for synchronous gRPC calls, leaving asynchronous and HTTP communications untraced. Additionally, earlier drafts of asynchronous tracing attempted to modify gRPC channels after creation, which violated the immutability rules of the underlying Python gRPC library, and did not consistently forward client options across all transport classes.

Solutions

This pull request provides a unified, cross-transport tracing implementation:

1. Universal 4-Transport Support:

   • Synchronous gRPC (GrpcTransport): Continues using OpenTelemetry gRPC channel interceptors.
   • Asynchronous gRPC (GrpcAsyncIOTransport): Supplies OpenTelemetry interceptors directly during channel creation, respecting the immutable design of asynchronous gRPC channels.
   • Synchronous REST (RestTransport): Adds wire span tracking around HTTP requests with automatic W3C trace context header injection (traceparent).
   • Asynchronous REST (AsyncRestTransport): Integrates HTTP wire span tracking and async context lifecycle handling with W3C header propagation.

2. Refined Transport Contracts & Cleanup:

   • Replaced per-instance signature inspection with module-level constants to eliminate overhead during method invocation.
   • Configured BaseTransport.kind to safely return an empty string by default instead of raising an exception.
   • Updated REST and base transport initializers to accept and store client options, ensuring options are forwarded uniformly across all transport classes.
   • Resolved an issue in asynchronous method wrappers where caller-specified compression settings were not correctly evaluated.
   • Added task cancellation handling (asyncio.CancelledError) so spans are closed accurately when asynchronous tasks are cancelled.

Notes for Reviewers

• This pull request targets template definitions in packages/gapic-generator and core helper functions in packages/google-api-core.
• Per team coordination, generated client library code and integration goldens are not included in this change and will be updated in a follow-up step.
• All additions maintain backward compatibility and preserve zero overhead when tracing is disabled.

[gemini-code-assist]

Code Review

This pull request implements OpenTelemetry tracing capabilities across gRPC and REST transports for generated clients. It introduces _observability modules, updates transport constructors to accept client_options, and wires tracing interceptors into the transport layers. The reviewer identified a critical issue in the _shared_macros.j2 template where start_http_span was called with incorrect arguments, which would cause a runtime TypeError. The suggested fix involves passing a properly constructed request object to the tracing utility.

[gemini-code-assist]

The current call to _observability.start_http_span passes client_options as the first positional argument (which maps to request in the function signature) and passes method, url, headers, and body as keyword arguments. However, start_http_span does not accept these keyword arguments and expects a request object with those attributes. This will result in a TypeError at runtime when tracing is enabled.

To fix this, dynamically construct a lightweight request object using Python's built-in type() constructor and pass it as the request argument, while correctly passing url_template and client_options as keyword arguments.

        with _observability.start_http_span(
            type("Request", (), {"method": method, "url": url, "headers": headers, "body": body})(),
            url_template=uri,
            client_options=client_options,
        ) as span:

[chalmerlowe]

Note

As context for the reviewer:

For native methods (like Echo or GetSecret), the generator reads the service's .proto file directly, so constructing the name in the template is straightforward:

method_name="{{ '.'.join(method.meta.address.package) }}.{{ service.name }}/{{ method.name }}"

However, mixins don't live in the service’s proto. Mixin methods (GetOperation, GetIamPolicy, ListLocations) are synthetic—they are injected by the generator from the static catalog in gapic/schema/mixins.py

Without a name attribute, any mixin call (like polling an operation or checking IAM permissions) would be unable to start an OpenTelemetry method span, or would emit an unknown/nameless span that failed our contract checks.

[chalmerlowe]

The focus for this file is to get fundamental/core elements into the Base class to:

• Help eliminate some checks that we were originally considering placing in the Transport classes, etc.
• Cut down on some of the boilerplate in the Transport classes

[chalmerlowe]

Comment for Reviewers:

This logic got moved to the base transport.