Skip to content

minor: Register Dart enabled historicals via a new DartWorkerService which Dart controller uses for discovering workers - #20508

Merged
gianm merged 1 commit into
apache:masterfrom
capistrant:DartWorkerService
Oct 8, 2026
Merged

gianm merged 1 commit into
apache:masterfrom
capistrant:DartWorkerService

Conversation

@capistrant

@capistrant capistrant commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

Description

Instead of coupling the Dart worker to the Historical service, I think it makes sense to tie an announced DartWorkerService component to historicals who enable Dart engine via runtime property. If nothing else it adds some more rich information to the historicals announced capabilities. But it also could provide us future optionality in how Dart workers are registered and used. The recommended guidance in dart docs continues to recommend setting the dart flag in common runtime properties; so early adopters who already do that will see zero impact or behavior change as long as they follow the rolling update order of historicals before brokers in the Druid 39 upgrade.

Mostly things will stay exactly the same, but now the Dart controller will only enroll actual Dart workers instead of all historicals without first checking if they run Dart in the first place. This would avoid the controller from sending Dart queries to a historical who explicitly disabled Dart.

Release note

Dart workers on historicals are announced and discovered by the broker with a new DartWorkerService mechanism. Operators whose clusters have enabled dart should be sure to follow the standard rolling upgrade/downgrade order where historicals restart before broker during upgrade and the reverse during downgrade.


Key changed/added classes in this PR
  • DartControllerContext
  • DartMessageRelays
  • DartWorkerService

This PR has:

  • been self-reviewed.
  • added documentation for new or modified features or behaviors.
  • a release note entry in the PR description.
  • added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
  • added or updated version, license, or notice information in licenses.yaml
  • added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
  • added unit tests or modified existing tests to cover new code paths, ensuring the threshold for code coverage is met.
  • added integration tests.
  • been tested in a test Druid cluster.

… of inferring workers solely off service == historical
@github-actions github-actions Bot added Area - Batch Ingestion Area - MSQ For multi stage queries - https://github.com/apache/druid/issues/12262 labels Oct 7, 2026
@capistrant capistrant changed the title minor: Discover Dart enabled historicals via a new DartWorkerService which Dart controller uses for discovering workers minor: Register Dart enabled historicals via a new DartWorkerService which Dart controller uses for discovering workers Oct 7, 2026

@FrankChen021 FrankChen021 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The controller now enrolls workers from the advertised Dart capability and uses the same filtered discovery for message relays. The inventory and discovery views use matching advertised host-and-port identifiers; I found no actionable issues in the changed behavior.

Reviewed 9 of 9 changed files (6 production files and 3 test files), plus the relevant discovery, inventory, service-announcement, and CLI module wiring.

Validation: git diff --check 131989817271bc836e815db738cc2be33fba3538 a970f6c2de7e94ee3140995fe2b154eae7371927 passed. No tests or builds were run; this was a static review.


This is an automated review by Codex GPT-5.6-Luna(max)

@gianm
gianm merged commit 3cedc8a into apache:master Oct 8, 2026
30 checks passed
@github-actions github-actions Bot added this to the 39.0.0 milestone Oct 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Area - Batch Ingestion Area - MSQ For multi stage queries - https://github.com/apache/druid/issues/12262

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants