refactor(phase-5): dispatcher abstraction + controller dissolution - #58
Closed
grubmeshi wants to merge 2 commits into
Closed
Conversation
…bt, adapt plan Runs every A1-A12 verification step from PLAN_DETAIL_05_dispatcher.md §1 against the phase-4-single-binary tree. A3 (k8s Job/Secret/SA manifest goldens via kubernetes/fake), A4 (Engine.Execute unification), A5 (shared meshapi.DecryptRunDetails/NodeId), A6 (zero package-level mutable state) and A8 (8-line coverage gate incl. crypto) are materially or partially false — the same plan-2/3 DDD-refactor debt phase 3's and phase 4's own STOP-gate checks already found and named, now a third consecutive check to record it. Resolved autonomously without halting, triaged per finding rather than blanket-deferred: A3 is genuinely blocking (kubernetes.go has zero test coverage and its concrete *kubernetes.Clientset field blocks fake-clientset tests outright) so step 3 is expanded to interface-ify the clientset field and author the manifest goldens fresh; A4 is reinterpreted (the tf handler wraps the existing, correctly-shaped SingleRunWorker.ExecuteRun rather than requiring a new Engine type); A5's decryption.go package-fate row is corrected (moves to internal/k8sjob, not meshapi); A6/A8 are accepted, non-blocking debt carried forward with the rest of the pattern. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Dissolve internal/controller into two D11 packages and introduce the dispatcher framework that the standalone in-process runners will consume: - internal/dispatch: generalized claim/drain Loop (Dispatcher/ClaimedRun/ Claimer/StatusApi/ClaimClassifier seam), the InProcess dispatcher (go-func per run, synchronous in-flight increment, Done() wake, Wait() drain within a configurable grace), the RunHandler interface + in-process secret/auth contract, Capability parsing (5 concrete types + ALL), the two dispatcher-authored UnhandledTypeError messages (frozen k8sjob text byte-identical to controller.go:201; new actionable InProcess text), and the relocated run_controller_* MetricsCollector (names/labels frozen). - internal/k8sjob: KubernetesJobDispatcher (Job/Secret/SA manifests, RunTooLargeError, CountActiveJobs, per-run decryption before Job creation), clientset widened to kubernetes.Interface (A3), fresh kubernetes/fake goldens, cluster I/O isolated in cluster.go (excluded), WIF registration DTO builder. Config split per plan §5. - internal/controller deleted; cmd/bbrunner run-controller rewired onto dispatch.Loop + k8sjob.KubernetesJobDispatcher (k8s dispatch path behavior-verbatim). controllerConfig embeds k8sjob.Config yaml:",inline". - Concurrency-hazard suite H1-H8 under -race; tf TfBinaries concurrent-GetTF test (H1). loop.shutdownCalled bool -> atomic.Bool (B6 data-race fix). - Gate extension: dispatch + k8sjob join thresholds at 90 (97.7% / 95.7%); k8sjob/cluster.go excluded; depguard groups replace the controller group. Sanctioned delta (§12): maxConcurrentJobs compiled-in default 20 -> 10. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Collaborator
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Phase 5 — Dispatcher Abstraction & In-Process Concurrency
Stacked on
phase-4-single-binary. Consolidates four parallel slices into onealways-green commit. All gates pass:
task test(-race),task lint(0 issues),
task coverage(dispatch 97.7%, k8sjob 95.7%, all others ≥90%).What landed
internal/dispatch— generalized claim/drainLoop(Dispatcher/ClaimedRun/Claimer/StatusApi/ClaimClassifierseam),InProcessdispatcher (go-func per run, synchronous in-flight increment before spawn,
Done()wake,Wait()drain within configurable grace),RunHandlerinterface + in-process secret/auth contract,
Capabilityparsing, the twoUnhandledTypeErrormessages (frozen k8sjob text byte-identical, newactionable InProcess text), relocated
run_controller_*MetricsCollector.internal/k8sjob—KubernetesJobDispatchermoved from the formercontroller;
clientsetwidened tokubernetes.Interface(A3) with freshkubernetes/fakegoldens;cluster.goreal-cluster I/O isolated + excluded.internal/controllerdeleted;cmd/bbrunnerrun-controller rewired ontodispatch.Loop+k8sjob.KubernetesJobDispatcher— k8s dispatch pathbehavior-verbatim (claim wire, decryption order, Job/Secret/SA manifests,
registration PUT, metric names all preserved).
-race; tfTfBinariesconcurrentGetTFtest (H1).loop.shutdownCalledbool→atomic.Bool(B6 fix).dispatch+k8sjobat 90; depguard groups replace thecontroller group;
internal/controllerprometheus/lint exclusions removed.Sanctioned deltas (§12)
maxConcurrentJobscompiled-in default 20 → 10 (run-controllerpersona; field/semantics + unlimited backstop otherwise unchanged; operators
can still set any value). Realized:
k8sjob.DefaultMaxConcurrentJobs = 10.NOT delivered by the parallel slices (scope gap — see uncertainties)
The dispatcher framework,
InProcess,RunHandler,Capabilityand the k8scontroller dissolution are complete and consumed by the controller persona.
The tf-persona cutover was not delivered (plan steps 5, 6, 8, 9):
tf.NewHandler(dispatch.RunHandler) does not exist;runTfPolling(
cmd/bbrunner/tf.go) andcmd/tfstill run the oldtf.NewManagerpollingloop + shared
SetRunToken/ClearRunTokenslot.maxConcurrentRunsconfig (default 3, delta feat: Improves ENV pass-through #5), the tfregistration:section,
runner_runs_unhandled_total/runner_at_capacity_skips_totaladditive metrics, and
cmd/bbrunnerdispatcher auto-detect are absent.LoopDepshas noWakechannel field (§17 promise);InProcess.Done()exists but is wired into no production Loop.
Consequence: the tf persona is unchanged (safest state — no frozen tf
contract touched), but the phase's headline in-process concurrency is not yet
live.
InProcess/RunHandler/Capabilityship as tested-but-dormant librarycode ready for that cutover. Exit criteria for steps 6/8/9 are therefore not
met; the interface shapes phase 6 builds against (§17) are present.
🤖 Generated with Claude Code