[Bug]: BigQueryIO.read() requires GCS tempLocation when using BigQuery Emulator #35777

Activity

1. hichmo

   added

   bug

   awaiting triage

   on Aug 3, 2025
2. github-actions

   added

   java

   io

   P2

   on Aug 3, 2025
3. Abacn commented on Aug 8, 2025

   

   Abacn

   on Aug 8, 2025

   Contributor

   More actions

   This is WAI. BigQueryIO.read() default to EXPORT, where Beam does not read from BigQuery directly. It invokes BigQuery export API to export tables to gcs, then Beam just read GCS files.

   Have you tried BigQueryIO.read with DIRECT_READ method (which does not require tempLocation)?
4. hichmo commented on Aug 8, 2025

   

   hichmo

   on Aug 8, 2025

   Author

   More actions

   I have also try with DIRECT_READ and face another issue.

   The BigQuery job completes successfully, but the Apache Beam pipeline remains stuck, beam is likely waiting for the results to be available in the temporary table
5. github-actions commented on Jan 6, 2026

   

   github-actionsBot

   on Jan 6, 2026 – with GitHub Actions

   Contributor

   More actions

   This issue has been marked as stale due to 150 days of inactivity. It will be closed in 30 days if no further activity occurs. If you think that’s incorrect or this issue still needs to be addressed, please simply write any comment. If closed, you can reopen the issue at any time. Thank you for your contributions.
6. github-actions

   added

   stale

   on Jan 6, 2026
7. github-actions commented on Feb 5, 2026

   

   github-actionsBot

   on Feb 5, 2026 – with GitHub Actions

   Contributor

   More actions

   This issue has been closed due to lack of activity. If you think that is incorrect, you can reopen the issue at any time.
8. github-actions

   closed this as not plannedon Feb 5, 2026
9. Abacn

   added

   pinned

   and removed

   stale

   awaiting triage

   on Feb 6, 2026
10. Abacn

    reopened this on Feb 6, 2026
11. mxtymoshyk commented on Aug 31, 2026

    

    mxtymoshyk

    on Aug 31, 2026

    Last edited by mxtymoshyk

    Contributor

    More actions

    @Abacn thanks for reopening this. I reproduced it against ghcr.io/goccy/bigquery-emulator and the DIRECT_READ half that @hichmo raised in the follow-up looks like a real gap, distinct from the tempLocation part.

    The tempLocation half is WAI, as you said. BigQueryIO.java:1415 already skips that check when the method is DIRECT_READ, so nothing needs to change there:

    if (getMethod() != TypedRead.Method.DIRECT_READ) {
      String tempLocation = bqOptions.getTempLocation();
      checkArgument(!Strings.isNullOrEmpty(tempLocation), "BigQueryIO.Read needs a GCS temp location ...");

    Why DIRECT_READ hangs

    StorageClientImpl honors --bigQueryEndpoint but keeps the default gRPC transport, which negotiates TLS (BigQueryServicesImpl.java:1902-1913):

    BigQueryReadSettings.Builder settingsBuilder =
        BigQueryReadSettings.newBuilder()
            .setCredentialsProvider(FixedCredentialsProvider.create(options.getGcpCredential()))
            .setTransportChannelProvider(
                BigQueryReadSettings.defaultGrpcTransportProviderBuilder()
                    .setHeaderProvider(USER_AGENT_HEADER_PROVIDER)
                    .build())
            .setReadRowsRetryAttemptListener(listener);
    @Nullable String endpoint = options.getBigQueryEndpoint();
    if (!Strings.isNullOrEmpty(endpoint)) {
      settingsBuilder.setEndpoint(trimSchemaIfNecessary(endpoint));
    }

    The emulator's gRPC port is plaintext, so the handshake never completes. A/B with the same client, same endpoint, only the transport differing:

    Transport	Result
    InstantiatingGrpcChannelProvider + setChannelConfigurator(c -> c.usePlaintext())	success in 0.4s, read session created, streams=1
    BigQueryReadSettings.defaultGrpcTransportProviderBuilder() (what Beam uses)	hangs; killed at 25s and again at 60s

    Parked here:

    com.google.cloud.bigquery.storage.v1.BigQueryReadClient.createReadSession(BigQueryReadClient.java:259)
    com.google.api.gax.rpc.ApiExceptions.callAndTranslateApiException(ApiExceptions.java:53)
    

    The emulator logged nothing at all for the TLS attempt, i.e. the request never got past the handshake. The plaintext row is the control: the emulator's Storage Read API itself works.

    In a real pipeline this is worse than the 25s above suggests. BigQueryServicesImpl.java:1918-1925 sets createReadSession's initial, max and total RPC timeout to Duration.ofHours(2), so the user-visible symptom is a silent two-hour stall rather than an error. That matches @hichmo's report that the BigQuery job completes but the pipeline stays stuck.

    newBigQueryWriteClient (BigQueryServicesImpl.java:1786-1789) has the same gap, so STORAGE_WRITE_API against an emulator is blocked in the same way.

    A second, independent blocker: one option, two ports

    Running an actual DIRECT_READ pipeline never even reaches the storage client. With --bigQueryEndpoint=http://localhost:9060/ (the value the gRPC client needs), the REST client is pointed at the gRPC port and hangs there first:

    BigQueryServicesImpl$DatasetServiceImpl.getTable(BigQueryServicesImpl.java:703)
    BigQueryServicesImpl.executeWithRetries(BigQueryServicesImpl.java:1720)
    BigQueryStorageTableSource.getTargetTable(BigQueryStorageTableSource.java:220)
    

    bigQueryEndpoint feeds three clients with incompatible requirements: the REST client takes it verbatim as setRootUrl and needs http://host:9050/, while both gRPC clients run it through trimSchemaIfNecessary (BigQueryServicesImpl.java:1803) and need host:9060. The emulator serves REST on 9050 and gRPC on 9060, so no single value works. Fixing plaintext alone would not be enough.

    Precedent already in this module

    FirestoreStatefulComponentFactory.java:86-97 solves exactly this, in the same package tree:

    if (emulatorHostPort != null) {
      builder
          .setCredentialsProvider(FixedCredentialsProvider.create(new EmulatorCredentials()))
          .setEndpoint(emulatorHostPort)
          .setTransportChannelProvider(
              InstantiatingGrpcChannelProvider.newBuilder()
                  .setEndpoint(emulatorHostPort)
                  .setChannelConfigurator(c -> c.usePlaintext())
                  .build());
    }

    SpannerAccessor.java:275-277 does the same behind a usePlaintext flag.

    One question before I write anything

    Any fix here has to add public surface to BigQueryOptions, and that is the part I did not want to guess at. What reads most naturally to me is a new getBigQueryEmulatorHost(): it would point the two gRPC clients at that host with plaintext and a no-op credential, and leave bigQueryEndpoint alone for the REST client. That follows the Firestore precedent above, and since it is purely additive nothing changes for anyone already setting bigQueryEndpoint. The other way to do it would be a separate gRPC endpoint option alongside an explicit plaintext flag, which is closer to the Spanner shape.

    Do you have a preference? I have the reproduction set up and am happy to send a PR either way. Not claiming the issue in the meantime.

    Setup for all of the above: Beam 2.75.0 on the DirectRunner, emulator seeded with a single three-row table.

    One note if you try to reproduce this on 2.75.0. Before reaching either of the blockers above you will hit an NPE in BigQueryStorageTableSource.getEstimatedSizeBytes, because the emulator returns no numBytes for the table. That one is unrelated to this issue and is already fixed on master by 4e5ae91.