Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .nextchanges/bundles/accurate-resource-summary-counts.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
* Bundle `deploy` and `destroy` summaries now report how many resources actually succeeded and, on a partial failure, how many failed, instead of the planned count. For example, if one of three resources fails to delete, the summary now reads `Destroy: 2 deleted, 1 failed` rather than `Destroy: 3 deleted`.
1 change: 1 addition & 0 deletions .nextchanges/bundles/print-destroy-summary-on-error.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
* Bundle destroy now prints the deletion summary even when destroy encounters errors, since partial deletions may have already succeeded.
16 changes: 16 additions & 0 deletions acceptance/bundle/deploy/summary-on-error/databricks.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
bundle:
name: test-deploy-summary-on-error

resources:
jobs:
# Chained job_01 <- job_02 <- job_03 via id references, so deploy creates them in a
# fixed order (job_01, job_02, job_03). Independent jobs would create in parallel,
# making both the faulted create and the error output vary per run.
job_01:
name: job-01
job_02:
name: job-02
description: needs ${resources.jobs.job_01.id}
job_03:
name: job-03
description: needs ${resources.jobs.job_02.id}
2 changes: 2 additions & 0 deletions acceptance/bundle/deploy/summary-on-error/out.test.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
Cloud = false
EnvMatrix.DMS = ["", "true"]
18 changes: 18 additions & 0 deletions acceptance/bundle/deploy/summary-on-error/output.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@

=== Deploy three chained jobs; the second create fails
>>> fault.py POST /api/2.2/jobs/create 400 1 1 INVALID_PARAMETER_VALUE

>>> musterr [CLI] bundle deploy
Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/test-deploy-summary-on-error/default/files...
Created jobs.job_01
Error: cannot create resources.jobs.job_02: Fault injected by test. (400 INVALID_PARAMETER_VALUE)

Endpoint: POST [DATABRICKS_URL]/api/2.2/jobs/create
HTTP Status: 400 Bad Request
API error_code: INVALID_PARAMETER_VALUE
API message: Fault injected by test.

Error: cannot create resources.jobs.job_03: dependency failed: resources.jobs.job_02

Files: 1 uploaded, 0 deleted
Resources: 1 created, 0 changed, 0 deleted, 0 unchanged, 2 failed
6 changes: 6 additions & 0 deletions acceptance/bundle/deploy/summary-on-error/script
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
title "Deploy three chained jobs; the second create fails"
# Creates run in chain order (job_01, job_02, job_03); faulting the second create makes
# job_01 succeed, job_02 fail, and job_03 skip because its dependency failed. The summary
# reports the accurate counts (1 created, 2 failed) rather than the planned 3 created.
trace fault.py "POST /api/2.2/jobs/create" 400 1 1 INVALID_PARAMETER_VALUE
trace musterr $CLI bundle deploy
4 changes: 4 additions & 0 deletions acceptance/bundle/deploy/summary-on-error/test.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
# fault.py registers its rule on the fake server, which a real workspace does not have.
Cloud = false

Ignore = [".databricks"]
16 changes: 16 additions & 0 deletions acceptance/bundle/destroy/summary-on-error/databricks.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
bundle:
name: test-destroy-summary-on-error

resources:
jobs:
# Chained job_01 <- job_02 <- job_03 via id references, so destroy deletes them in a
# fixed order (children first: job_03, job_02, job_01). Independent jobs would delete
# in parallel, making both the faulted delete and the error output vary per run.
job_01:
name: job-01
job_02:
name: job-02
description: needs ${resources.jobs.job_01.id}
job_03:
name: job-03
description: needs ${resources.jobs.job_02.id}
2 changes: 2 additions & 0 deletions acceptance/bundle/destroy/summary-on-error/out.test.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
Cloud = false
EnvMatrix.DMS = ["true"]
29 changes: 29 additions & 0 deletions acceptance/bundle/destroy/summary-on-error/output.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@

=== Deploy three chained jobs
>>> [CLI] bundle deploy
Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/test-destroy-summary-on-error/default/files...
Created jobs.job_01
Created jobs.job_02
Created jobs.job_03
Files: 1 uploaded, 0 deleted
Resources: 3 created, 0 changed, 0 deleted, 0 unchanged

=== Destroy: only one delete fails, yet the summary counts every planned deletion
>>> fault.py POST /api/2.2/jobs/delete 400 2 1 INVALID_PARAMETER_VALUE

>>> musterr [CLI] bundle destroy --auto-approve
The following resources will be deleted:
delete resources.jobs.job_01
delete resources.jobs.job_02
delete resources.jobs.job_03

All files and directories at the following location will be deleted: /Workspace/Users/[USERNAME]/.bundle/test-destroy-summary-on-error/default

Error: cannot delete resources.jobs.job_01: deleting id=[JOB_01_ID]: Fault injected by test. (400 INVALID_PARAMETER_VALUE)

Endpoint: POST [DATABRICKS_URL]/api/2.2/jobs/delete
HTTP Status: 400 Bad Request
API error_code: INVALID_PARAMETER_VALUE
API message: Fault injected by test.

Destroy: 2 deleted, 1 failed
13 changes: 13 additions & 0 deletions acceptance/bundle/destroy/summary-on-error/script
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
title "Deploy three chained jobs"
trace $CLI bundle deploy
# Mask the job IDs (the destroy error below prints job_01's) so the golden is stable.
replace_ids.py

title "Destroy: only one delete fails, yet the summary counts every planned deletion"
# Deletes run children-first (job_03, job_02, job_01); faulting the third makes only
# job_01 fail while job_03 and job_02 are really deleted. logDestroySummary runs from a
# defer, so the summary still prints on the error return (without the defer the error
# would skip it). The count is the planned deletions, not the ones that succeeded, so
# job_01 is reported as deleted although it was not. An accurate partial count is a follow-up.
trace fault.py "POST /api/2.2/jobs/delete" 400 2 1 INVALID_PARAMETER_VALUE
trace musterr $CLI bundle destroy --auto-approve
8 changes: 8 additions & 0 deletions acceptance/bundle/destroy/summary-on-error/test.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# fault.py registers its rule on the fake server, which a real workspace does not have.
Cloud = false

# A failed resource delete only surfaces as a destroy error under the DMS (deployment
# history) engine, so pin that variant to exercise the error path deterministically.
EnvMatrix.DMS = ["true"]

Ignore = [".databricks"]
1 change: 1 addition & 0 deletions acceptance/bundle/dms/failed-delete/output.txt
Original file line number Diff line number Diff line change
Expand Up @@ -112,6 +112,7 @@ HTTP Status: 400 Bad Request
API error_code: INVALID_PARAMETER_VALUE
API message: Fault injected by test.

Destroy: 0 deleted, 1 failed

>>> print_requests.py --dms //api/2.0/bundle
{
Expand Down
1 change: 1 addition & 0 deletions acceptance/bundle/dms/failed-recreate/output.txt
Original file line number Diff line number Diff line change
Expand Up @@ -114,6 +114,7 @@ API error_code: INVALID_PARAMETER_VALUE
API message: Fault injected by test.

Files: 2 uploaded, 0 deleted
Resources: 0 created, 0 changed, 0 deleted, 0 unchanged, 1 failed

>>> print_requests.py --dms //api/2.0/bundle
{
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,7 @@ API error_code: PERMISSION_DENIED
API message: Fault injected by test.

Files: 1 uploaded, 0 deleted
Resources: 0 created, 0 changed, 0 deleted, 1 unchanged, 1 failed

=== The service lists the job with the failure
>>> MSYS_NO_PATHCONV=1 [CLI] bundle-deployments list-resources deployments/[DEPLOYMENT_ID] -o json
Expand Down
1 change: 1 addition & 0 deletions acceptance/bundle/dms/record-failure/output.txt
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ API error_code: INVALID_PARAMETER_VALUE
API message: Node type nonexistent-node-type is not supported. Supported node types: [NODE_TYPES]

Files: 2 uploaded, 0 deleted
Resources: 0 created, 0 changed, 0 deleted, 0 unchanged, 1 failed

>>> [CLI] bundle plan -o json
{
Expand Down
1 change: 1 addition & 0 deletions acceptance/bundle/migrate/auto/apply-failure/output.txt
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ API error_code: INJECTED
API message: Fault injected by test.

Files: 1 uploaded, 0 deleted
Resources: 0 created, 0 changed, 0 deleted, 0 unchanged, 1 failed

Exit code: 1

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,3 +11,4 @@ Error: cannot create resources.jobs.bar: dependency failed: resources.jobs.foo
Error: cannot create resources.jobs.baz: dependency failed: resources.jobs.foo

Files: 0 uploaded, 0 deleted
Resources: 1 created, 0 changed, 0 deleted, 0 unchanged, 3 failed
9 changes: 8 additions & 1 deletion acceptance/bundle/resource_deps/create_error/output.txt
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,14 @@ create jobs.foo

Plan: 3 to add, 0 to change, 0 to delete, 1 unchanged

=== Expecting no difference in the output between first and second deploy
=== Second deploy: only the summary differs, independent is now unchanged--- out.deploy.txt
+++ out.deploy2.txt
@@ -12,3 +12,3 @@

Files: 0 uploaded, 0 deleted
-Resources: 1 created, 0 changed, 0 deleted, 0 unchanged, 3 failed
+Resources: 0 created, 0 changed, 0 deleted, 1 unchanged, 3 failed

>>> print_requests.py --nostamp //jobs
{
"method": "POST",
Expand Down
4 changes: 3 additions & 1 deletion acceptance/bundle/resource_deps/create_error/script
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,9 @@ trace $CLI bundle plan

rm "$OUT_REQUESTS"
musterr $CLI bundle deploy -q &> out.deploy2.txt
title "Expecting no difference in the output between first and second deploy"
# The only difference is the summary: the first deploy created the independent job (1
# created), the second finds it unchanged. foo still fails and bar/baz still skip on both.
title "Second deploy: only the summary differs, independent is now unchanged"
diff.py out.deploy.txt out.deploy2.txt
rm out.deploy2.txt
trace print_requests.py --nostamp //jobs
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,5 +3,6 @@ Created pipelines.foo
Error: cannot create resources.pipelines.bar: cannot resolve "${resources.pipelines.foo.ingestion_definition.connection_name}": ingestion_definition: cannot access nil value

Files: 4 uploaded, 0 deleted
Resources: 1 created, 0 changed, 0 deleted, 0 unchanged, 1 failed

Exit code: 1
Original file line number Diff line number Diff line change
Expand Up @@ -3,3 +3,4 @@ Created jobs.test
Error: cannot create resources.jobs.bar: cannot resolve "${resources.jobs.test.tasks[0].new_cluster.custom_tags.missing_tag}": tasks[0].new_cluster.custom_tags.missing_tag: key "missing_tag" not found in map

Files: 4 uploaded, 0 deleted
Resources: 1 created, 0 changed, 0 deleted, 0 unchanged, 1 failed
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@ API error_code: RESOURCE_ALREADY_EXISTS
API message: An app with the same name already exists: test-app-already-exists

Files: 2 uploaded, 0 deleted
Resources: 0 created, 0 changed, 0 deleted, 0 unchanged, 1 failed

>>> [CLI] apps delete test-app-already-exists
{
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,3 +9,4 @@ API error_code: INVALID_PARAMETER_VALUE
API message: Invalid input: RPC CreateCatalog Field managedcatalog.CatalogInfo.name: name "" is not a valid name. Valid names cannot contain spaces, periods, forward slashes, or control characters.

Files: 2 uploaded, 0 deleted
Resources: 0 created, 0 changed, 0 deleted, 0 unchanged, 1 failed
Original file line number Diff line number Diff line change
Expand Up @@ -9,3 +9,4 @@ API error_code: INVALID_PARAMETER_VALUE
API message: policy_family_id and definition cannot be used together

Files: 2 uploaded, 0 deleted
Resources: 0 created, 0 changed, 0 deleted, 0 unchanged, 1 failed
Original file line number Diff line number Diff line change
Expand Up @@ -7,5 +7,6 @@ API error_code: RESOURCE_DOES_NOT_EXIST
API message: Warehouse doesnotexist does not exist

Files: 5 uploaded, 0 deleted
Resources: 0 created, 0 changed, 0 deleted, 0 unchanged, 1 failed

Exit code: 1
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ API error_code: INJECTED
API message: Fault injected by test.

Files: 1 uploaded, 0 deleted
Resources: 0 created, 0 changed, 0 deleted, 0 unchanged, 1 failed

Exit code: 1

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ API error_code: INJECTED
API message: Fault injected by test.

Files: 1 uploaded, 0 deleted
Resources: 1 created, 0 changed, 0 deleted, 0 unchanged, 1 failed

Exit code: 1

Expand Down
2 changes: 2 additions & 0 deletions acceptance/bundle/resources/job_runs/failed_run/output.txt
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ run page: [RUN_URL]
Error: cannot create resources.jobs.downstream_job: dependency failed: resources.job_runs.my_run

Files: 3 uploaded, 0 deleted
Resources: 1 created, 0 changed, 0 deleted, 0 unchanged, 2 failed

=== the failed run is recorded, and not having succeeded is drift
>>> read_id.py my_run
Expand Down Expand Up @@ -65,6 +66,7 @@ run page: [RUN_URL]
Error: cannot create resources.jobs.downstream_job: dependency failed: resources.job_runs.my_run

Files: 0 uploaded, 0 deleted
Resources: 0 created, 0 changed, 0 deleted, 1 unchanged, 2 failed

=== run-now was issued once per deploy, and the recreate left the failed run alone
>>> print_requests.py --keep //jobs/run-now
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ API error_code: INJECTED
API message: Fault injected by test.

Files: 1 uploaded, 0 deleted
Resources: 1 created, 0 changed, 0 deleted, 0 unchanged, 1 failed

Exit code: 1

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ API error_code: INVALID_PARAMETER_VALUE
API message: Cluster validation error: Validation failed for custom_tags, CostCenter must be from-policy (is "not-what-the-policy-says")

Files: 3 uploaded, 0 deleted
Resources: 1 created, 0 changed, 0 deleted, 0 unchanged, 1 failed

>>> [CLI] bundle destroy --auto-approve
The following resources will be deleted:
Expand Down
2 changes: 2 additions & 0 deletions acceptance/bundle/resources/jobs/create-error/output.txt
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ API error_code: INVALID_PARAMETER_VALUE
API message: Shared job cluster feature is only supported in multi-task jobs.

Files: 1 uploaded, 0 deleted
Resources: 0 created, 0 changed, 0 deleted, 0 unchanged, 1 failed

>>> musterr [CLI] bundle deploy --force-lock -q
Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/test-bundle/default/files...
Expand All @@ -20,6 +21,7 @@ API error_code: INVALID_PARAMETER_VALUE
API message: Shared job cluster feature is only supported in multi-task jobs.

Files: 0 uploaded, 0 deleted
Resources: 0 created, 0 changed, 0 deleted, 0 unchanged, 1 failed

>>> musterr [CLI] bundle deploy --force-lock -qq
Error: cannot create resources.jobs.foo: Shared job cluster feature is only supported in multi-task jobs. (400 INVALID_PARAMETER_VALUE)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,5 +13,6 @@ API error_code: INVALID_PARAMETER_VALUE
API message: git_source.git_provider must be one of: github,gitlab,bitbucketcloud,gitlabenterpriseedition,bitbucketserver,azuredevopsservices,githubenterprise,awscodecommit

Files: 3 uploaded, 0 deleted
Resources: 0 created, 0 changed, 0 deleted, 0 unchanged, 1 failed

Exit code: 1
Original file line number Diff line number Diff line change
Expand Up @@ -9,3 +9,4 @@ API error_code: INVALID_PARAMETER_VALUE
API message: Got an invalid name ''. Registered Model names cannot be empty strings.

Files: 3 uploaded, 0 deleted
Resources: 0 created, 0 changed, 0 deleted, 0 unchanged, 1 failed
Original file line number Diff line number Diff line change
Expand Up @@ -12,3 +12,4 @@ API error_code: INVALID_PARAMETER_VALUE
API message: Field 'branch_id' is required, expected non-default value (not "")!

Files: 0 uploaded, 0 deleted
Resources: 1 created, 0 changed, 0 deleted, 0 unchanged, 1 failed
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ API error_code: INVALID_PARAMETER_VALUE
API message: Field database_id must match pattern ^[a-z]([a-z0-9-]{0,61}[a-z0-9])?$, got 'Invalid_DB_ID'.

Files: 0 uploaded, 0 deleted
Resources: 3 created, 0 changed, 0 deleted, 0 unchanged, 1 failed

>>> [CLI] bundle destroy --auto-approve
The following resources will be deleted:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ API error_code: NOT_FOUND
API message: role not found; role_id:"does-not-exist" [TraceId: [TRACE_ID]]

Files: 0 uploaded, 0 deleted
Resources: 2 created, 0 changed, 0 deleted, 0 unchanged, 1 failed

>>> [CLI] bundle destroy --auto-approve
The following resources will be deleted:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,5 +7,6 @@ API error_code: INVALID_PARAMETER_VALUE
API message: Field 'spec.suspension' is in update_mask but not provided in request

Files: 0 uploaded, 0 deleted
Resources: 0 created, 0 changed, 0 deleted, 2 unchanged, 1 failed

Exit code: 1
Original file line number Diff line number Diff line change
Expand Up @@ -13,3 +13,4 @@ API error_code: INVALID_PARAMETER_VALUE
API message: Field 'endpoint_id' is required, expected non-default value (not "")!

Files: 0 uploaded, 0 deleted
Resources: 2 created, 0 changed, 0 deleted, 0 unchanged, 1 failed
Original file line number Diff line number Diff line change
Expand Up @@ -7,5 +7,6 @@ API error_code: INVALID_PARAMETER_VALUE
API message: Field 'spec.default_endpoint_settings.suspension' is in update_mask but not provided in request

Files: 0 uploaded, 0 deleted
Resources: 0 created, 0 changed, 0 deleted, 0 unchanged, 1 failed

Exit code: 1
Original file line number Diff line number Diff line change
Expand Up @@ -7,5 +7,6 @@ API error_code: INVALID_PARAMETER_VALUE
API message: Field 'spec.history_retention_duration' is in update_mask but not provided in request

Files: 0 uploaded, 0 deleted
Resources: 0 created, 0 changed, 0 deleted, 0 unchanged, 1 failed

Exit code: 1
Original file line number Diff line number Diff line change
Expand Up @@ -11,3 +11,4 @@ API error_code: INVALID_PARAMETER_VALUE
API message: Field 'project_id' is required, expected non-default value (not "")!

Files: 0 uploaded, 0 deleted
Resources: 0 created, 0 changed, 0 deleted, 0 unchanged, 1 failed
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ API error_code: BAD_REQUEST
API message: role with that name already exists

Files: 0 uploaded, 0 deleted
Resources: 0 created, 0 changed, 0 deleted, 2 unchanged, 1 failed

>>> [CLI] bundle destroy --auto-approve
The following resources will be deleted:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/recreate-capped-po
Error: cannot recreate resources.postgres_synced_tables.my_table: waiting after deleting id=synced_tables/lakebase_test_[UNIQUE_NAME].public.trips_synced: timed out: synced table still exists, waiting for deletion to complete

Files: 0 uploaded, 0 deleted
Resources: 0 created, 0 changed, 0 deleted, 3 unchanged, 1 failed

Exit code: 1

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ API error_code: RESOURCE_ALREADY_EXISTS
API message: Vector search endpoint with name vs-endpoint-b-[UNIQUE_NAME] already exists

Files: 0 uploaded, 0 deleted
Resources: 0 created, 0 changed, 0 deleted, 1 unchanged, 1 failed

Exit code: 1

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,3 +10,4 @@ API error_code: INVALID_PARAMETER_VALUE
API message: CreateVolume storage_location can not be provided.

Files: 3 uploaded, 0 deleted
Resources: 1 created, 0 changed, 0 deleted, 0 unchanged, 1 failed
Loading
Loading