Skip to content

Implement package layering refactor (Phases 1-2) - #806

Merged
bschwedler merged 4 commits into
mainfrom
refactor/package-layering-plan
Sep 30, 2026
Merged

bschwedler merged 4 commits into
mainfrom
refactor/package-layering-plan

Conversation

@bschwedler

@bschwedler bschwedler commented Sep 17, 2026 •

Copy link
Copy Markdown
Contributor

Break circular dependencies in posit-bakery by splitting config layer into separate build, targets, and registry management packages.

Phase 1: Extract build execution and registry cleanup from BakeryConfig into dedicated build/ and registry_management/ packages. Moves build_targets, bake_plan_json, metadata loading/merging, clean_caches, and clean_temporary out of config layer.

Phase 2: Extract target selection logic into targets/ package. Consolidates duplicate filtering logic from cli/ci.py::matrix with BakeryConfig.generate_image_targets into a single select_targets function. Moves BakerySettings and BakeryConfigFilter to config/settings.py.

Result: config/ no longer imports image, parallel, or registry_management. The BakeryConfig.targets compatibility property was removed to avoid recreating the config → targets → image → config cycle; callers now select targets explicitly with targets.select_targets(config, settings).

Dependency rule after split:

  • cli, plugins → build, targets, registry_management, config, image, parallel
  • build → targets, image, parallel, config
  • targets → config, image
  • registry_management → image
  • image → config, parallel
  • config → config, const, error, util, settings

Plan: package-layering.md

Related issue:

@github-actions

github-actions Bot commented Sep 17, 2026 •

Copy link
Copy Markdown

Test Results

2 339 tests  +4   2 339 ✅ +4   9m 5s ⏱️ + 1m 22s
    1 suites ±0       0 💤 ±0 
    1 files   ±0       0 ❌ ±0 

Results for commit 367b368. ± Comparison against base commit 2a5bc5a.

♻️ This comment has been updated with latest results.

@bschwedler
bschwedler force-pushed the refactor/package-layering-plan branch 2 times, most recently from 1e28edb to 7c8260d Compare September 23, 2026 13:24
@bschwedler

bschwedler commented Sep 23, 2026 •

Copy link
Copy Markdown
Contributor Author

Package dependencies after the split:

flowchart LR
  cli <--> plugins
  cli --> build & targets & registry_management
  plugins --> build & targets
  build --> image
  targets --> image
  registry_management --> image
  image --> config & parallel
Loading

Edges implied by a longer path are omitted (e.g. cli → config). config no longer imports image, parallel, or registry_management, which breaks the config ↔ image cycle. The cli ↔ plugins cycle predates this PR: cli loads plugins through plugins.registry, and plugins import shared CLI helpers from cli.common.

@bschwedler

bschwedler commented Sep 23, 2026 •

Copy link
Copy Markdown
Contributor Author

Moved functions

Paths are relative to posit_bakery/.

config/config.py → build/runner.py

# before
def BakeryConfig.build_targets(
    self,
    load: bool = True,
    push: bool = False,
    pull: bool = False,
    cache: bool = True,
    platforms: list[str] | None = None,
    strategy: ImageBuildStrategy = ImageBuildStrategy.BAKE,
    metadata_file: Path | None = None,
    fail_fast: bool = False,
    retry: int = 0,
    jobs: int | None = None,
)

# after
def build_targets(
    base_path: Path,
    targets: list[ImageTarget],
    temp_registry: str | None,
    clean_temporary: bool,
    load: bool = True,
    push: bool = False,
    pull: bool = False,
    cache: bool = True,
    platforms: list[str] | None = None,
    strategy: ImageBuildStrategy = ImageBuildStrategy.BAKE,
    metadata_file: Path | None = None,
    fail_fast: bool = False,
    retry: int = 0,
    jobs: int | None = None,
) -> BuildResult

config/config.py → build/runner.py (renamed)

# before
def BakeryConfig.bake_plan_targets(self, push: bool = False) -> str

# after
def bake_plan_json(
    base_path: Path, targets: list[ImageTarget], push: bool = False
) -> str

config/config.py → build/runner.py

# before
def BakeryConfig.load_build_metadata_from_file(
    self, metadata_file: Path
) -> list[str]

# after
def load_build_metadata_from_file(
    targets: list[ImageTarget], metadata_file: Path
) -> list[str]

config/config.py → build/runner.py

# before
def BakeryConfig._merge_sequential_build_metadata_files(
    self,
) -> dict[str, Any]

# after
def _merge_sequential_build_metadata_files(
    targets: list[ImageTarget],
) -> dict[str, Any]

config/config.py → registry_management/clean.py

clean_temporary has the same signature change.

# before
def BakeryConfig.clean_caches(
    self,
    remove_untagged: bool = True,
    remove_older_than: timedelta | None = None,
    dry_run: bool = False,
)

# after
def clean_caches(
    targets: list[ImageTarget],
    remove_untagged: bool = True,
    remove_older_than: timedelta | None = None,
    dry_run: bool = False,
) -> list[Exception]

config/config.py → targets/selection.py (renamed)

# before
def BakeryConfig.generate_image_targets(
    self, settings: BakerySettings = BakerySettings()
)

# after
def select_targets(
    config: "BakeryConfig", settings: BakerySettings, *, sort: bool = True
) -> list[ImageTarget]

cli/common.py → targets/selection.py

# before
def exit_if_no_targets(config: "BakeryConfig", settings: "BakerySettings") -> None

# after
def exit_if_no_targets(
    targets: list[ImageTarget], settings: BakerySettings, context: str = "build"
) -> None

(new) → targets/ci_matrix.py

# before: inline loop in the `ci matrix` command, no function

# after
def matrix_rows(
    targets: list[ImageTarget], exclude: list[BakeryCIMatrixFieldEnum] | None = None
) -> list[dict[str, str | bool]]

config/config.py → deleted

# before
def BakeryConfig.get_image_target_by_uid(
    self, uid: str
) -> ImageTarget | None

# after: removed; publish() now filters select_targets() output by UID

Moved with no signature change

  • _retry_build: config/config.py → build/runner.py
  • apply_recent_versions: config/config.py → targets/selection.py (type hints now quoted)
  • _describe_active_filters: cli/common.py → targets/selection.py
  • version_matches: config/config.py → config/image/parsed_version.py
  • BakerySettings.dev_stream, .effective_dev_channel, .migrate_dev_stream_to_dev_channel: BakerySettings moved from config/config.py → config/settings.py

@bschwedler
bschwedler marked this pull request as ready for review September 23, 2026 17:41
@bschwedler
bschwedler force-pushed the refactor/package-layering-plan branch 2 times, most recently from 4605d2d to 66c3358 Compare September 23, 2026 17:53
Split build execution, target selection, and registry cleanup out of
BakeryConfig. Pass configuration and targets explicitly to reduce
coupling and break the config/image dependency cycle.

- Move build orchestration and metadata handling to build/runner.py
- Move target selection and CI matrix generation to targets/
- Move registry cleanup to registry_management/clean.py, collapsing
  clean_caches/clean_temporary into a shared helper
- Thread the per-command context (build/test/lint/scan/tag) through
  exit_if_no_targets so its error message names the right command
- Keep CLI and plugin behavior covered by the refactored test suite
@bschwedler
bschwedler force-pushed the refactor/package-layering-plan branch from 66c3358 to 207e33e Compare September 23, 2026 18:29

@ianpittwood ianpittwood left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The bulk of this looks good, just a few things we may want to address.

Comment thread posit-bakery/posit_bakery/plugins/builtin/dgoss/__init__.py Outdated
Comment thread posit-bakery/posit_bakery/plugins/builtin/hadolint/__init__.py Outdated
"""
return _clean_registries(
targets,
registry_name=lambda target: cn.split(":")[0] if (cn := target.cache_name()) else None,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there's some risk with the else None fallback. I don't see any filtering done later in this logic so there's an opportunity for a TypeError at REGISTRY_PATTERN.match() calls.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I factored this out to a helper for easier cognition and exception handling:

def _cache_registry_name(target: ImageTarget) -> str | None:
"""Return the registry from a target's cache name, if it has one."""
cache_name = target.cache_name()
if not cache_name:
return None
return cache_name.split(":", maxsplit=1)[0]

Skip targets without cache names before invoking GHCR cleanup.\n\nExtract registry-name resolution into a helper to make the missing-cache case explicit.
Use the module name in the no-target error context for consistent identification.
@bschwedler
bschwedler force-pushed the refactor/package-layering-plan branch from d101e54 to bd46a51 Compare September 30, 2026 15:14
* main:
  Fold long table cell content instead of truncating
  Widen default Rich console width for non-TTY output
  build(deps): bump anyio
  Rename deprecated GitHub-hosted runner labels
  chore: ignore .pi-subagents, .rpiv, .serena
  build(deps): bump the actions group across 3 directories with 8 updates
  build(deps): bump the python-deps group across 2 directories with 3 updates
  Package bakery skill as plugin
@bschwedler
bschwedler added this pull request to the merge queue Sep 30, 2026
Merged via the queue into main with commit d2f095c Sep 30, 2026
29 checks passed
@bschwedler
bschwedler deleted the refactor/package-layering-plan branch September 30, 2026 15:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Split BakeryConfig into a document model, CRUD manager, and build service

2 participants