Skip to content

O11Y-982: Move non-interactive BigQuery queries to batch jobs#3576

Draft
djwhitt wants to merge 5 commits into
mainfrom
davidwhittington/o11y-982-logflare-querying-move-some-queries-over-to-batched-queueing
Draft

O11Y-982: Move non-interactive BigQuery queries to batch jobs#3576
djwhitt wants to merge 5 commits into
mainfrom
davidwhittington/o11y-982-logflare-querying-move-some-queries-over-to-batched-queueing

Conversation

@djwhitt

@djwhitt djwhitt commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

Summary

  • add a BigQuery batch query path using jobs.insert with BATCH priority and polling via getQueryResults
  • route alert queries, endpoint proactive refreshes, and UI tailing polling queries through batch jobs
  • update tests for batch execution paths

Tests

  • mix test test/logflare/bigquery/bq_repo_test.exs test/logflare/alerting_test.exs test/logflare/alerting/alert_worker_test.exs test/logflare/backends/adaptor/incidentio_adaptor_test.exs test/logflare/logs/search_operations_test.exs test/logflare/endpoints/cache_test.exs

Linear: O11Y-982

djwhitt and others added 4 commits June 9, 2026 09:42
The `{:ok, %Job{jobReference: job_reference}}` clause matches any
`{:ok, %Job{}}` (binding a possibly-nil reference), so the explicit
`{:ok, %Job{}}` clause could never be reached. Nil references are
already handled by poll_batch_query_results.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant