Repository navigation
Epic: split pkg/compose into focused internal packages #14396
Description
Context
pkg/compose is one flat package: about 62 non-test files and 19.5k lines, where every operation is a method of the same composeService (214 methods over 46 files). Any operation can therefore reach anything else — the Docker client, the event bus, the prompt, any other operation. Nothing but convention keeps a read-only command from depending on lifecycle code, and every test builds a whole service.
The package also exports about 60 top-level symbols, among them internals (Plan, Operation, ObservedState, Graph, the *Hash helpers). The module is /v5: every exported symbol is a compatibility commitment, and the more of the engine lives in the public package, the harder it is to change.
This epic tracks two linked goals, one reviewable PR at a time:
- splitting
pkg/composeinto focused packages underinternal/; - reducing its exported surface, before SDK callers build on symbols that were never meant to be public. The two go together: moving a cluster into
internal/is also what stops its types from being public API, so the public-surface items below are tracked here rather than separately.
Approach
The pattern settled by the first moves:
- each cluster becomes its own
internal/package, starting with the most isolated ones; - it declares what it needs from the engine as a small
Backendinterface, and cannot reach anything else (importingpkg/composeback would be an import cycle); composeServiceimplements that interface and keeps one-line forwarders, so the callers of the moved code don't change;- behavior is unchanged, and tests move with the code with the same assertions.
The lifecycle clusters (the plan engine, up, down, …) come last, once the start-phase series of #14081 has landed.
Progress
Preparation and first moves
- Stop
pkg/composefrom importing the CLI: the interactive menu ofupbecomes an injected interface. → refactor(up): inject the navigation menu instead of importing cmd/formatter #14324 (merged) - Move the read-only operations (
ps,ls,top,port,volumes,events,viz) tointernal/inspect, and project loading and generation tointernal/projectloader. → refactor: extract inspect and projectloader from pkg/compose into internal packages #14331 (merged) - Move the provider plugin execution and the relay it may deploy to
internal/providerandinternal/relay. → refactor: move the provider relay and plugin execution to internal/relay and internal/provider #14333 (merged) - Move
exec,cp,exportandcommittointernal/containerops, and settle how the terminal crosses the boundary (through the Docker CLI theBackendexposes). → refactor: move exec, cp, export and commit to internal/containerops #14397 (merged)
Remaining moves
The order is a proposal and each item is to be confirmed against what it needs from composeService before it starts.
- Extract the shared kernel: the small helpers (progress events, container naming, …) that
pkg/composestill keeps its own copies of next to the ones ininternal/progressandinternal/naming. - Images:
pull,push,imagesand image pruning. -
publish, together withpkg/compose/transform. -
build(classic and bake). -
logsandattach. - The plan engine (
plan,reconcile,observed_state,executor), after the start-phase series of Epic: converge the start phase into the plan engine #14081. - The lifecycle commands (
up,down,start,stop,restart,run, …) on top of the engine.
Public surface
What pkg/compose exports today is a compatibility commitment of the /v5 module for anything that doesn't need to be. These items shrink it, and are best done together with the moves above, since a package that moves to internal/ no longer exports its types.
- Take the engine's types (
Plan,Operation,ObservedState,Graph) and theServiceHash/NetworkHash/VolumeHashhelpers out of the exported API ofpkg/compose. - Fix or remove
WithDryRun: it is exported but takes the unexported*composeService, so it can't be used from outside the package. - Move
pkg/compose/transformunderinternal/: it is exported but onlypublishuses it. - Move
pkg/e2eout ofpkg/: its framework and scenario code (framework.go,scenario.go,checks.go,assert.go) is non-test code published underpkg/.
Non-goals
- No behavior change: a move PR is a move. Fixes found on the way go in their own PR.
- No new public API: what a package needs from the engine goes through its
Backendinterface, not through an exported symbol ofpkg/compose.
Activity
refactor(containerops): move exec, cp, export and commit to internal/…
test(containerops): pin which containers cp targets