Skip to content

air: bound submission config validation timeout and retry - #6994

Open
caroline-db wants to merge 1 commit into
mainfrom
air/preflight-validation-timeout
Open

caroline-db wants to merge 1 commit into
mainfrom
air/preflight-validation-timeout

Conversation

@caroline-db

@caroline-db caroline-db commented Oct 8, 2026 •

Copy link
Copy Markdown
Contributor

Changes

  • Limit normal databricks air run submission preflight validation to one request with a five-second deadline and no retries.
  • Warn and continue for unavailable validation: disabled or missing endpoints, HTTP 408/429/5xx, transport failures, deadlines, and invalid success responses.
  • Continue blocking field errors, caller and authentication failures, request-construction failures, and user cancellation.

Why

Normal submission previously used the standard SDK client. The SDK could retry HTTP 429, HTTP 503, transport failures, and recognized transient errors for the configured retry window, which defaults to five minutes. After retries ended, the CLI failed open only for disabled or missing endpoints and 5xx responses; terminal HTTP 429 and transport failures still blocked submission.

Jobs and AICM validate authoritatively after preflight, so validation-service availability should not delay or block normal submission. Definitive configuration and caller errors still stop before artifact upload and Jobs submission.

Tests

  • Cover one-attempt behavior for HTTP and transport failures, the five-second bound, availability classification, warnings, caller errors, malformed responses, and cancellation.
  • go test ./cmd/air
  • go vet ./cmd/air
  • ./task checks
  • bodyclose: 0 issues
  • Full local lint is blocked by a golangci-lint analyzer panic in dependency package poll; full local tests pass cmd/air but unrelated JSON diagnostic-position assertions fail under the local Go toolchain.

This PR was written with Codex.

@github-actions github-actions Bot added the AIR Databricks AI Runtime CLI label Oct 8, 2026
@caroline-db caroline-db changed the title air: bound submission config validation to five seconds air: bound submission config validation timeout and retry Oct 8, 2026
@caroline-db
caroline-db force-pushed the air/preflight-validation-timeout branch from 8bf0bbd to 7d03020 Compare October 8, 2026 18:38
@eng-dev-ecosystem-bot

Copy link
Copy Markdown
Collaborator

Integration test report

Commit: 7d03020

Run: 37825886640

Env 🔄​flaky ✅​pass 🙈​skip Time
✅​ aws linux-2core-8gb 276 16 5:41
✅​ aws-windows-latest-4core-16gb 278 14 3:51
✅​ azure linux-2core-8gb 275 16 4:50
🔄​ azure-windows-latest-4core-16gb 1 276 14 4:05
✅​ gcp linux-2core-8gb 276 16 5:19
✅​ gcp-windows-latest-4core-16gb 278 14 3:59
Test Name azure-windows-latest-4core-16gb
🔄​ TestFilerWorkspaceFilesExtensionsDelete 🔄​f
Top 6 slowest tests (at least 2 minutes):
duration env testname
3:56 gcp-windows-latest-4core-16gb TestAccept
3:50 gcp linux-2core-8gb TestAccept
3:50 aws-windows-latest-4core-16gb TestAccept
3:47 aws linux-2core-8gb TestAccept
3:47 azure-windows-latest-4core-16gb TestAccept
3:04 azure linux-2core-8gb TestAccept

@caroline-db
caroline-db marked this pull request as ready for review October 8, 2026 22:41
@caroline-db
caroline-db requested review from a team as code owners October 8, 2026 22:41
@caroline-db
caroline-db requested a review from rugpanov October 8, 2026 22:41
@rugpanov
rugpanov removed their request for review October 9, 2026 08:39

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

AIR Databricks AI Runtime CLI

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants