[Failing Test]: BigqueryClient.queryUnflattened fails BigQuery ITs on a transient 503 #40792

beam_PreCommit_Java_GCP_IO_Direct_BigQuery failed on #40790 in StorageApiDataTriggeredSchemaUpdateIT.testDataTriggeredSchemaUpgradeExactlyOnce:

com.google.api.client.googleapis.json.GoogleJsonResponseException: 503 Service Unavailable
GET https://bigquery.googleapis.com/bigquery/v2/projects/apache-beam-testing/queries/job_...
"reason": "backendError"
"message": "Error encountered during execution. Retrying may solve the problem."
    at org.apache.beam.sdk.io.gcp.testing.BigqueryClient.queryUnflattened(BigqueryClient.java:345)
    at org.apache.beam.sdk.io.gcp.bigquery.StorageApiDataTriggeredSchemaUpdateIT.verifyDataWritten(StorageApiDataTriggeredSchemaUpdateIT.java:367)

The pipeline had finished, and the test failed while checking its output. BigqueryClient.queryUnflattened polls jobs.getQueryResults with no retry, so a single 503 fails the test. The other BigqueryClient methods, such as createNewDataset, createNewTable and queryWithRetries, already retry with backoff. 12 test files call queryUnflattened.

queryUnflattened also leaves its temporary dataset behind when the query returns no rows or throws.

Failed job: https://github.com/apache/beam/actions/runs/40205636577/job/121205844102