Skip to content

[Prism] Wait for the job service before starting loopback workers - #40773

Open
bvolpato wants to merge 1 commit into
apache:masterfrom
bvolpato:bvolpato/prism-job-service-readiness
Open

bvolpato wants to merge 1 commit into
apache:masterfrom
bvolpato:bvolpato/prism-job-service-readiness

Conversation

@bvolpato

Copy link
Copy Markdown
Contributor

Related #40298.

Prism's port reservation is released before the child process binds it. The loopback worker can take that port in the meantime. CI then logs bind: address already in use, followed by UNIMPLEMENTED ...JobService/Prepare from the wrong service (example).

Wait for a successful JobService.GetJobs response before starting the portable runner. Retry failed startup on a fresh port, stopping the failed child and closing its channel. Preserve interruption during cleanup. Retries apply only to service startup.

Reproduced the port conflict with a competing gRPC service. The startup regressions fail before the fix and pass afterward. Module tests, targeted loopback tests, formatting, Checkstyle, and dependency checks passed. The module tests can be run with ./gradlew :runners:prism:java:test.

@github-actions github-actions Bot added runners prism size/m M: 50-249 lines changed labels Oct 25, 2026
@bvolpato
bvolpato marked this pull request as ready for review October 25, 2026 05:38

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📋 Review Summary

This PR resolves a port reservation race condition between Prism's child process and loopback workers by verifying JobService readiness via GetJobs before starting the PortableRunner, with bounded retries and proper process/channel cleanup on failure. The implementation is well-structured, handles thread interruption cleanly, and includes thorough unit and regression test coverage.

🔍 General Feedback

  • The readiness check in PrismRunner (probing the TCP socket before invoking JobService.GetJobs to avoid gRPC reconnect backoff, and rejecting non-retryable gRPC status codes like UNIMPLEMENTED if another service bound the port) directly and cleanly addresses the race condition.
  • Resource cleanup and thread interruption preservation in PrismExecutor.stop() and PrismRunner.startPrism() are handled properly.
  • Consider cleaning up the duplicate dependencies block in runners/prism/java/build.gradle in a follow-up.

💡 Tip: Comment /gemini review on this pull request to trigger a new Gemini code review.

@github-actions

Copy link
Copy Markdown
Contributor

Assigning reviewers:

R: @tvalentyn added as fallback since no labels match configuration
Backup reviewer: jrmccluskey (if the main reviewer is unresponsive, you can escalate to them by tagging their username)

Note: If you would like to opt out of this review, comment assign to next reviewer.

Available commands:

  • stop reviewer notifications - opt out of the automated review tooling
  • remind me after tests pass - tag the comment author after tests pass
  • waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

The PR bot will only process comments in the main thread (not review comments).

@codecov

codecov Bot commented Oct 25, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 0.00%. Comparing base (a8d6afd) to head (8bdc964).
⚠️ Report is 75 commits behind head on master.

Additional details and impacted files
@@             Coverage Diff             @@
##           master   #40773       +/-   ##
===========================================
- Coverage   80.04%        0   -80.05%     
===========================================
  Files         490        0      -490     
  Lines       82916        0    -82916     
===========================================
- Hits        66373        0    -66373     
+ Misses      16543        0    -16543     
Flag Coverage Δ
python ?

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant