You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Part of #2648. Read the epic first: it holds the model and the decisions.
Goal
A target can cover the whole planet. A weekly refresh bakes only what changed.
Scope
OSM by size. A region reads the smallest source that covers it. Small regions read Geofabrik extracts; planet reads osm-planet, kept current with osm-replication daily diffs (osmium apply-changes). Both sources already exist in data/sources.toml.
Leaf extracts. Port the planet sharding of host/obc-bake/src/planet.rs into the internal OSM steps. A leaf extract must not carry the planet timestamp in its header. A leaf whose OSM content did not change keeps its digest, so its tiled layers are not baked again.
P7-C: a step whose key equals its output in the serving release is unchanged and is not baked, also on another machine; the run reports unchanged leaves; each child program's CPU and peak memory are measured on their own (wait4)
+382 / −62
Acceptance (a) now holds for a large Geofabrik region. After an OSM refresh, only the leaves whose OSM content changed bake again, and the run logs, for example, device/map: 3 of 40 leaves baked, 37 unchanged.
Next:
P7-A: the named partitions hook for whole layers (acceptance b).
P7-D: the planet source (osm-planet plus daily osm-replication diffs). It starts after P6-3b.
P7-E: delete obc bake --all. It waits for the owner.
P7-F: the bake machine requirements. They come after the owner's measurement run.
P7-A is merged: #2693 (+119 / −20), the named partitions hook.
A step of a Shape::Partitioned layer is named by its partition: the last part of its name. A step whose last part is not a declared partition is refused.
Each partition stales, bakes, is kept and publishes on its own, because the engine already works per step.
obc data --json gives partitions and stale_partitions per layer, and the review gives the partitions that bake.
a_partitioned_layer_stales_and_bakes_one_partition_alone covers acceptance (b): a new version of one partition's input stales only that partition, and only it bakes.
Both acceptance items now have their tests (a: #2688 and #2690, b: #2693). Two scope items are still open:
the planet source (P7-D, after P6-3b);
the bake machine requirements (P7-F, after the owner's measurement run).
So this issue stays open. obc bake --all (P7-E) waits for the owner's answer.
P7-D (the planet source) is reviewed as draft #2702, head ee3c188. It is stacked on #2701 and merges after it.
Region kind planet (data/regions/planet.toml). It resolves to B = osm-planet and E = osm-replication with from=B.
One shared step, device/source/planet, runs osmium apply-changes … --generator obc-data. When E has no diff after B, the step links the planet. The step has no client file. Another machine builds it again from the R2 copies of the two fetches.
osm-planet refreshes every 28 days. The dead "before osm-planet" rule is deleted.
The planner refuses a planet region. This is the recommended answer to Q1, which the owner has not decided yet. An update now refuses before it fetches data when a product does not suit the settings, so a planet update does not download 95 GB first. That rule applies to every update.
The store-safe hard link is one function, engine::link, used by every step that links a store object.
The PR body lists the planet-scale risks to measure: apply-changes RAM, peak disk, step length, manifest and catalog size, GLO-30 downloads, the per-leaf manifest parse, and the diff copies of about 3.7 GB a week.
P7-E (delete obc bake --all) waits for the owner's answer to Q6. P7-F waits for the owner's measurement run (Q7).
This issue is merged into #3000 (world scale). The acceptance tests landed (#2688, #2690, #2693), and coverage is now Europe (#2921 A6). Draft PR #2702 and Q7 are noted in #3000. Q6 goes to #2656.
Activity
timohueser commentedon Oct 12, 2026
Progress on P7. These PRs are merged:
obc-osm(at most two extracts per Osmium run),--generator obc-data, Osmium in the Rust CI test jobwait4)Acceptance (a) now holds for a large Geofabrik region. After an OSM refresh, only the leaves whose OSM content changed bake again, and the run logs, for example,
device/map: 3 of 40 leaves baked, 37 unchanged.Next:
planetsource (osm-planetplus dailyosm-replicationdiffs). It starts after P6-3b.obc bake --all. It waits for the owner.Generated by Claude Code
timohueser commentedon Oct 12, 2026
P7-A is merged: #2693 (+119 / −20), the named partitions hook.
Shape::Partitionedlayer is named by its partition: the last part of its name. A step whose last part is not a declared partition is refused.obc data --jsongivespartitionsandstale_partitionsper layer, and the review gives the partitions that bake.a_partitioned_layer_stales_and_bakes_one_partition_alonecovers acceptance (b): a new version of one partition's input stales only that partition, and only it bakes.Both acceptance items now have their tests (a: #2688 and #2690, b: #2693). Two scope items are still open:
planetsource (P7-D, after P6-3b);So this issue stays open.
obc bake --all(P7-E) waits for the owner's answer.Generated by Claude Code
timohueser commentedon Oct 13, 2026
P7-D (the planet source) is reviewed as draft #2702, head ee3c188. It is stacked on #2701 and merges after it.
planet(data/regions/planet.toml). It resolves to B =osm-planetand E =osm-replicationwithfrom=B.device/source/planet, runsosmium apply-changes … --generator obc-data. When E has no diff after B, the step links the planet. The step has no client file. Another machine builds it again from the R2 copies of the two fetches.osm-planetrefreshes every 28 days. The dead "before osm-planet" rule is deleted.planetregion. This is the recommended answer to Q1, which the owner has not decided yet. An update now refuses before it fetches data when a product does not suit the settings, so a planet update does not download 95 GB first. That rule applies to every update.engine::link, used by every step that links a store object.P7-E (delete
obc bake --all) waits for the owner's answer to Q6. P7-F waits for the owner's measurement run (Q7).Generated by Claude Code
timohueser commentedon Nov 4, 2026
This issue is merged into #3000 (world scale). The acceptance tests landed (#2688, #2690, #2693), and coverage is now Europe (#2921 A6). Draft PR #2702 and Q7 are noted in #3000. Q6 goes to #2656.
Generated by Claude Code