Skip to content

Preserve GCE bulk creation progress on timeout - #2194

Merged
micafer merged 5 commits into
apache:trunkfrom
ishu7w:preserve-gce-nodes-on-create-timeout
Nov 6, 2026
Merged

micafer merged 5 commits into
apache:trunkfrom
ishu7w:preserve-gce-nodes-on-create-timeout

Conversation

@ishu7w

@ishu7w ishu7w commented Oct 11, 2026

Copy link
Copy Markdown
Contributor

Description

Fixes #1589. When ex_create_multiple_nodes times out after some nodes have been created, its generic exception discards all progress. Callers cannot retrieve known results to recover or clean up their cloud resources.

Raise GCEMultipleNodesTimeoutError, a LibcloudError subclass, carrying nodes (known successful or failed results, in request order) and pending_node_names (results still unknown). Include the timeout value in the message. Existing successful calls still return the same list. Resources are not automatically deleted, and the documentation warns that pending operations can finish after the timeout.

Validation

  • Two regressions fail on unchanged trunk: timeout after mixed successful/failed/pending results, and timeout before any creation starts.
  • All 215 GCE and Google common connection/authentication tests pass, including existing bulk creation success tests.
  • Regression verifies retained object identity/order, unresolved names, timeout/driver details, and absence of automatic deletion.
  • Black, isort, flake8, and whitespace checks pass for changed Python files.
  • Tests use deterministic polling and provider fixtures; no live cloud resources were created.

Status

Ready for review.

Checklist

  • Code linting
  • Documentation
  • Tests
  • ICLA (if maintainers consider this change a bigger contribution)

AI assistance: This implementation, regression tests, and description were prepared with OpenAI Codex.

Expose completed node results and pending node names in a specific LibcloudError so callers can recover or clean up resources after a timeout. Pending operations may still complete; no resources are automatically deleted.

Fixes apache#1589
@ishu7w

ishu7w commented Oct 21, 2026

Copy link
Copy Markdown
Contributor Author

Gentle follow-up on this PR from October 11. GitHub currently reports no checks and there has not been review feedback yet. Could a maintainer let me know whether CI should be run or if any changes are needed?

@micafer micafer left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

This is a useful improvement to ex_create_multiple_nodes(). Preserving completed results and unresolved node names in a dedicated LibcloudError subclass gives callers the information they need to recover from partial creation failures.

The implementation is straightforward, and the regression tests cover both a mixed successful/failed/pending scenario and a timeout before creation starts.

One minor documentation suggestion: pending_node_names can also include nodes whose creation was never initiated (as demonstrated by the timeout=0 test), not only operations that may still be running. It might be worth clarifying this distinction in the exception docstring.

Otherwise, this looks good to merge.

@codecov-commenter

codecov-commenter commented Nov 5, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 83.58%. Comparing base (71900a8) to head (a7a22e7).

Additional details and impacted files
@@            Coverage Diff             @@
##            trunk    #2194      +/-   ##
==========================================
+ Coverage   83.57%   83.58%   +0.01%     
==========================================
  Files         352      352              
  Lines       82082    82117      +35     
  Branches     8802     8808       +6     
==========================================
+ Hits        68594    68634      +40     
+ Misses      10612    10608       -4     
+ Partials     2876     2875       -1     
Files with missing lines Coverage Δ
libcloud/compute/drivers/gce.py 77.66% <100.00%> (+0.20%) ⬆️
libcloud/test/compute/test_gce.py 97.80% <100.00%> (+0.03%) ⬆️
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@ishu7w

ishu7w commented Nov 6, 2026

Copy link
Copy Markdown
Contributor Author

Thanks, good point. I've updated the exception docstring to clarify that pending_node_names includes both nodes whose creation hasn't started and nodes with operations still in progress. It also explains that the latter may finish creating after the timeout.

The change is in bb773ba. I checked Python compilation and whitespace; this only changes the documentation.

@micafer
micafer merged commit 937dfdd into apache:trunk Nov 6, 2026
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

GCENodeDriver.ex_create_multiple_nodes doesn't clean itself when fails.

3 participants