[REFACTOR] Make sources.dataset optional

[Alex Iannicelli (atiannicelli)]

Summary

Changes the dataset field on the SourceItem model from required to optional, in both the Pydantic model and the JSON schema. Relaxing a constraint is a minor schema change, so this can ship ahead of the planned removal of dataset from the sources column.

• Resolves OvertureMaps/tf-data-platform#4684 (child of OvertureMaps/tf-data-platform#4683)
• Target release: 2026-09-23.0

Changes

• packages/overture-schema-common/src/overture/schema/common/sources.py — dataset moved from the required section to optional (str | None = None)
• schema/defs.yaml — sourcePropertyItem.required changed from [property, dataset] to [property]
• Regenerated all theme baseline schemas via make update-baselines (mechanical required list change only)
• packages/overture-schema-common/tests/test_models.py — expected JSON schema updated
• packages/overture-schema-codegen/tests/test_pyspark_scaffold.py — scaffold test updated: dataset is no longer populated as a required sibling

Testing

• pytest packages/ — 3059 passed (excluding pyspark suite, which has pre-existing environment-only Spark FileIO teardown warnings unrelated to this change; verified the same errors occur on a clean tree)
• Verified SourceItem(property="") now validates without dataset, and still accepts it when present
• Baselines regenerated and diff inspected — only the required list changed

Impact

• Backward compatible: all existing data with dataset remains valid
• Sources entries without dataset now pass validation

[github-actions]

🗺️ Schema reference docs preview is live!

🌍 Preview	https://staging.overturemaps.org/schema/pr/633/schema/index.html
🕐 Updated	Aug 12, 2026 13:33 UTC
📝 Commit	16a0911
🔧 env SCHEMA_PREVIEW	true

Note

♻️ This preview updates automatically with each push to this PR.

[Copilot]

Pull request overview

This PR relaxes the schema constraint for sources[].dataset by making it optional across the Python Pydantic model, the shared JSON Schema definitions, and all regenerated theme baseline schemas—supporting incremental refactors ahead of planned removal of dataset from the sources column.

Changes:

• Made SourceItem.dataset optional in the Pydantic model and updated the shared JSON schema definition to remove dataset from the required list.
• Regenerated and updated all theme baseline schemas to reflect that only "property" remains required for SourceItem.
• Updated tests to align expected JSON schema output and scaffold generation behavior with the new optionality.

Reviewed changes

Copilot reviewed 19 out of 19 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
schema/defs.yaml	Removes dataset from sourcePropertyItem required fields in shared schema defs.
packages/overture-schema-common/src/overture/schema/common/sources.py	Makes SourceItem.dataset optional (`str
packages/overture-schema-common/tests/test_models.py	Updates expected JSON schema to require only "property" for SourceItem.
packages/overture-schema-codegen/tests/test_pyspark_scaffold.py	Updates scaffold expectations: required "property" present; optional "dataset" omitted.
packages/overture-schema-transportation-theme/tests/segment_baseline_schema.json	Regenerated baseline: SourceItem.required drops "dataset".
packages/overture-schema-transportation-theme/tests/connector_baseline_schema.json	Regenerated baseline: SourceItem.required drops "dataset".
packages/overture-schema-places-theme/tests/place_baseline_schema.json	Regenerated baseline: SourceItem.required drops "dataset".
packages/overture-schema-divisions-theme/tests/division_boundary_baseline_schema.json	Regenerated baseline: SourceItem.required drops "dataset".
packages/overture-schema-divisions-theme/tests/division_baseline_schema.json	Regenerated baseline: SourceItem.required drops "dataset".
packages/overture-schema-divisions-theme/tests/division_area_baseline_schema.json	Regenerated baseline: SourceItem.required drops "dataset".
packages/overture-schema-buildings-theme/tests/building_part_baseline_schema.json	Regenerated baseline: SourceItem.required drops "dataset".
packages/overture-schema-buildings-theme/tests/building_baseline_schema.json	Regenerated baseline: SourceItem.required drops "dataset".
packages/overture-schema-base-theme/tests/water_baseline_schema.json	Regenerated baseline: SourceItem.required drops "dataset".
packages/overture-schema-base-theme/tests/land_use_baseline_schema.json	Regenerated baseline: SourceItem.required drops "dataset".
packages/overture-schema-base-theme/tests/land_cover_baseline_schema.json	Regenerated baseline: SourceItem.required drops "dataset".
packages/overture-schema-base-theme/tests/land_baseline_schema.json	Regenerated baseline: SourceItem.required drops "dataset".
packages/overture-schema-base-theme/tests/infrastructure_baseline_schema.json	Regenerated baseline: SourceItem.required drops "dataset".
packages/overture-schema-base-theme/tests/bathymetry_baseline_schema.json	Regenerated baseline: SourceItem.required drops "dataset".
packages/overture-schema-addresses-theme/tests/address_baseline_schema.json	Regenerated baseline: SourceItem.required drops "dataset".

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Victor Schappert (vcschapp)]

Alex Iannicelli (@atiannicelli) could you include a towncrier changelog fragment with this?

Sorry for this - we just merged the publishing workflow PR that requires a changelog fragment, so it's a bit of a surprise new requirement. The docs/versioning.md should explain how to do this.