Skip to content

[REFACTOR](build) Migrate workspace to uv_build and PEP 420 namespace packages - #623

Merged
Victor Schappert (vcschapp) merged 4 commits into
mainfrom
pep-420
Aug 12, 2026
Merged

[REFACTOR](build) Migrate workspace to uv_build and PEP 420 namespace packages#623
Victor Schappert (vcschapp) merged 4 commits into
mainfrom
pep-420

Conversation

@sethfitz

@sethfitz Seth Fitzsimmons (sethfitz) commented Jul 31, 2026

Copy link
Copy Markdown
Collaborator

Three commits.

refactor(build): migrate all packages from hatchling to uv_build

Switch every workspace package's build backend to uv_build. Because the packages share the overture.schema namespace, each declares an explicit [tool.uv.build-backend] module-name (e.g. overture.schema.common); name normalization would derive the wrong module. The overture-schema aggregator uses module-name = "overture.schema" with namespace = true and ships only overture/schema/py.typed — uv_build requires a module root, so a deps-only package still owns the namespace marker.

Convert the pkgutil namespaces to PEP 420: uv_build ships shared namespaces implicitly and never packages an overture/__init__.py, so the extend_path shims are removed. Wheels and editable installs both merge the namespace natively.

Deleting those __init__.py files removes the signal ruff's isort uses to detect overture as first-party (detect-same-package walks __init__ chains), so add src = ["packages/*/src"]. With overture now first-party everywhere, imports across the test suite and a few pyspark modules recanonicalize into their own group; that reformat is folded in here because it cannot land as a separately-green commit.

refactor(build): emit PEP 420 generated pyspark tree

Stop the pyspark codegen from emitting empty __init__.py files so the generated expression and test trees are PEP 420 namespace packages, matching the rest of the workspace. The mirrored layout (generated/overture/schema/<theme>/) is kept — only the init emission is dropped.

Removing the inits broke three things, each given a real fix:

  • pytest collection: generated conformance tests reached _support via deep relative imports. Under PEP 420 the module resolves as generated.*, so those imports overshoot the top level. Emit absolute from _support.X import ..., delete the _support_prefix machinery, and move the hand-written tests to the same form. Set --import-mode=importlib and consider_namespace_packages = true.
  • type checking: add the pyspark tests dir to mypy_path; keep the hand-written tests/__init__.py so pyspark's test_cli doesn't collide with codegen's under importlib.
  • runtime registry: pkgutil.walk_packages skips directories without __init__.py, finding zero generated modules. Walk the namespace roots as files instead; test_registry.py covers this — no existing test exercised the on-disk walk, so an empty registry would have passed silently.

chore(build): guard PEP 420 namespace roots against __init__.py

Add a check-namespace gate that fails if overture/ or overture/schema/ gains an __init__.py, which would turn a namespace root back into a regular package and shadow other workspace packages' contributions at import. Wired into the check gate so the PEP 420 invariant this stack establishes cannot silently regress.

Stacked on #620. Closes #618

@github-actions

github-actions Bot commented Jul 31, 2026 •
edited
Loading

Copy link
Copy Markdown

🗺️ Schema reference docs preview is live!
🌍 Preview https://staging.overturemaps.org/schema/pr/623/schema/index.html
🕐 Updated Aug 12, 2026 16:22 UTC
📝 Commit 674959e
🔧 env SCHEMA_PREVIEW true

Note

♻️ This preview updates automatically with each push to this PR.

lowlydba
John McCall (lowlydba) previously approved these changes Aug 11, 2026
View reviewed changes

@lowlydba John McCall (lowlydba) left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔥

John McCall (lowlydba) added a commit that referenced this pull request Aug 12, 2026
@lowlydba
[REFACTOR](ci) Un-hoist pyspark expression generation into the packag…
fff7195 
…e's own prebuild script

overture-schema-pyspark ships generated validation expressions that
aren't committed to git. main-publish.yaml and release-publish.yaml
special-cased it directly (if: ... == 'overture-schema-pyspark'
generate step, plus a post-build wheel-unzip check), so both workflows
had to know about one package's build quirk.

Moves that into packages/overture-schema-pyspark/scripts/prebuild.sh,
invoked generically as "run the package's prebuild.sh if it has one".
Neither workflow references pyspark by name anymore, and the script's
own empty-output guard replaces the wheel-content check. This also
keeps the convention backend-agnostic ahead of #623's hatchling ->
uv_build migration, which drops hatchling's custom build-hook support
entirely.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Signed-off-by: John McCall <john@overturemaps.org>
stack merge was automatically disabled August 12, 2026 16:15

Pull Request is not mergeable

stack merge was automatically disabled August 12, 2026 16:19

Pull Request is not mergeable

Base automatically changed from extract-validation-package to main August 12, 2026 16:20
@sethfitz
Seth Fitzsimmons (sethfitz) dismissed stale reviews from Victor Schappert (vcschapp) and John McCall (lowlydba) August 12, 2026 16:20

The merge-base changed after approval.

@sethfitz @vcschapp
refactor(build): migrate all packages from hatchling to uv_build
39f7db6 
Switch every workspace package's build backend to uv_build. Because the
packages share the overture.schema namespace, each declares an explicit
[tool.uv.build-backend] module-name (e.g. overture.schema.common); name
normalization would derive the wrong module. The overture-schema
aggregator uses module-name = "overture.schema" with namespace = true
and ships only overture/schema/py.typed -- uv_build requires a module
root, so a deps-only package still owns the namespace marker.

Convert the pkgutil namespaces to PEP 420: uv_build ships shared
namespaces implicitly and never packages an overture/__init__.py, so the
extend_path shims are removed. Wheels and editable installs both merge
the namespace natively.

Deleting those __init__.py files removes the signal ruff's isort uses to
detect overture as first-party (detect-same-package walks __init__
chains), so add src = ["packages/*/src"]. With overture now first-party
everywhere, imports across the test suite and a few pyspark modules
recanonicalize into their own group; that reformat is folded in here
because it cannot land as a separately-green commit.

Signed-off-by: Seth Fitzsimmons <seth@mojodna.net>
@sethfitz @vcschapp
refactor(build): emit PEP 420 generated pyspark tree
da72c78 
Stop the pyspark codegen from emitting empty `__init__.py` files so the
generated expression and test trees are PEP 420 namespace packages,
matching the rest of the workspace after the hatchling->uv_build
migration. The mirrored layout (generated/overture/schema/<theme>/) is
kept -- only the init emission is dropped.

Removing the inits broke three things the bead's plan assumed would keep
working; each needed a real fix:

- pytest collection: generated conformance tests reached `_support` via
  deep relative imports whose depth `_support_prefix` computed. Under
  PEP 420 the module resolves as `generated.*` (tests/ is on
  pythonpath), not `tests.generated.*`, so those relative imports
  overshoot the top level. Emit absolute `from _support.X import ...`
  instead and delete the `_support_prefix` machinery. To keep one import
  style across the test tree, the hand-written tests move to the same
  absolute form. Set `--import-mode=importlib` (via addopts; it has no
  ini form) and `consider_namespace_packages = true` so collection walks
  the namespace.

- type checking: mypy resolves `_support` as top-level only when its
  parent is a base, so add the pyspark tests dir to `mypy_path`. The
  hand-written `tests/__init__.py` stays: dropping it would make
  pyspark's `test_cli` a top-level module colliding with codegen's under
  importlib.

- runtime registry: `pkgutil.walk_packages` skips subdirectories without
  `__init__.py`, finding zero generated modules under the PEP 420 tree.
  Walk the namespace roots as files instead. test_registry.py covers
  this -- no existing test exercised the on-disk walk, so an empty
  registry would have passed the suite silently.

Signed-off-by: Seth Fitzsimmons <seth@mojodna.net>
@sethfitz @vcschapp
chore(build): guard PEP 420 namespace roots against __init__.py
77150e8 
check-namespace fails if overture/ or overture/schema/ gains an
__init__.py, which would turn a namespace root into a regular package
and shadow other workspace packages' contributions at import. Wired into
the check gate.

Signed-off-by: Seth Fitzsimmons <seth@mojodna.net>
@sethfitz @vcschapp
chore(build): add changelog fragments for uv_build/PEP 420 migration
674959e 
Signed-off-by: Seth Fitzsimmons <seth@mojodna.net>
@vcschapp
Victor Schappert (vcschapp) merged commit 293b395 into main Aug 12, 2026
25 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@vcschapp Victor Schappert (vcschapp) vcschapp approved these changes
@lowlydba John McCall (lowlydba) lowlydba approved these changes

Assignees

No one assigned

Labels

change type - cosmetic 🌹 Cosmetic change

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[CHORE] Migrate workspace to uv_build with PEP 420 namespace packages

3 participants

@sethfitz @vcschapp @lowlydba