[damccorm]

Summary

Fixes flaky/startup failures in PostCommit XVR JavaUsingPython Dataflow (e.g. run 42212921122) where the first wave of ValidateRunnerXlangTest test cases fail with UNAVAILABLE: io exception / Connection refused: localhost/127.0.0.1:<port> while the remaining 15 tests succeed once the Python expansion service finishes importing apache_beam and binds its port.

Root Cause

1. sdks/python/scripts/run_expansion_services.sh starts the Java and Python expansion services in the background (&) and exits immediately without waiting for them to bind to $JAVA_PORT and $PYTHON_PORT (and also leaks the flock file descriptor 200 to the child processes).
2. BaseExternalTest.waitForReady() built its ManagedChannel without .usePlaintext() (unlike External.java) and without calling channel.resetConnectBackoff() on TRANSIENT_FAILURE, and only polled for 15 seconds (30 * 500ms). Under CPU contention (8 parallel Gradle test workers starting simultaneously on Python 3.14), the Python expansion service can take ~35-40 seconds to finish importing apache_beam and bind the port, causing the first 5 test classes to fail right at the 15s mark.

Changes

• In sdks/python/scripts/run_expansion_services.sh, close FD 200 in backgrounded child processes and wait up to 60s for both expansion service ports to accept connections (or dump logs and fail fast if the process exits prematurely).
• In BaseExternalTest.waitForReady(), use .usePlaintext(), call channel.resetConnectBackoff() on TRANSIENT_FAILURE, and increase the readiness timeout from 15s to 60s.

[damccorm]

Closing to reopen from an upstream apache/beam branch with a verification workflow run.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 58.29%. Comparing base (3dd07de) to head (a022dd4).
⚠️ Report is 1 commits behind head on master.

Additional details and impacted files

@@             Coverage Diff              @@
##             master   #41034      +/-   ##
============================================
- Coverage     58.40%   58.29%   -0.11%     
  Complexity     2276     2276              
============================================
  Files          1136     1136              
  Lines        169395   169395              
  Branches       1482     1482              
============================================
- Hits          98937    98754     -183     
- Misses        67909    68092     +183     
  Partials       2549     2549              

Flag	Coverage Δ
python	80.17% <ø> (-0.22%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.