Skip to content

Add retry mechanism for transient network/IO failures in test execution - #367

Closed
MikeWLloyd wants to merge 1 commit into
askimed:mainfrom
MikeWLloyd:feature-#366
Closed

MikeWLloyd wants to merge 1 commit into
askimed:mainfrom
MikeWLloyd:feature-#366

Conversation

@MikeWLloyd

Copy link
Copy Markdown
Contributor

Summary

Add retry mechanism for transient network/IO failures.

When tests depend on remote files staged by Nextflow (e.g. from raw.githubusercontent.com), transient DNS or connection failures cause spurious test failures that are unrelated to the actual test logic. This PR adds a configurable retry mechanism that detects these failures and re-runs the affected test automatically.

What changed

• TestExecutionEngine: on test failure, reads the Nextflow stderr/stdout error report and checks it against a list of known transient error patterns. If a match is found and attempts remain, the test directories are re-initialized and the test is re-executed.
• Config: new retries DSL field (default 0) so retry count can be set in nf-test.config.
• RunTestsCommand: new --retries <N> CLI flag that overrides the config file value when provided.

Detected error patterns:

• Could not resolve host (DNS failure)
• Can't stage file https:// / No such file or directory: https:// (Nextflow staging failure)
• curl: (6) / curl: (7) / curl: (28) / curl: (35) (various curl network errors)
• Connection timed out / Connection refused

Usage
In nf-test.config:

config {
    retries = 3
}

Or on the CLI:

nf-test test --retries 3 tests/

CLI takes precedence over the config file value.

Testing

TestExecutionEngineRetryTest covers five cases using lightweight stubs (no external mocking library):
• retry succeeds before exhaustion → test passes, execute() called the expected number of times
• retries exhausted → test fails
• non-network error → no retry attempted
• retries=0 → no retry even on network error
• clean first-pass → execute() called exactly once

@MikeWLloyd MikeWLloyd closed this Aug 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant